2023-08-25T04:13:43.559 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-25T04:13:43.560 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} --archive /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064 --name rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/orig.config.yaml 2023-08-25T04:13:43.625 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T04:13:43.715 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064 branch: wip-rishabh-2023aug3-b5 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} email: null first_in_suite: false job_id: '7379064' kernel: client: branch: testing flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023aug3-b5 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023aug3-b5 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023aug3-b5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-25_01:50:32 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-25T04:13:43.715 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa; will attempt to use it 2023-08-25T04:13:43.716 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks 2023-08-25T04:13:43.717 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-25T04:13:43.719 INFO:teuthology.task.internal:Checking packages... 2023-08-25T04:13:43.750 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'ba6b39e4f4f77fb95c747099b53db0467d435f56' 2023-08-25T04:13:43.750 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-25T04:13:43.750 INFO:teuthology.packaging:ref: None 2023-08-25T04:13:43.751 INFO:teuthology.packaging:tag: None 2023-08-25T04:13:43.751 INFO:teuthology.packaging:branch: wip-rishabh-2023aug3-b5 2023-08-25T04:13:43.751 INFO:teuthology.packaging:sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:13:43.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023aug3-b5 2023-08-25T04:13:43.962 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5703.gba6b39e4 2023-08-25T04:13:43.962 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-25T04:13:43.969 INFO:teuthology.task.internal:no buildpackages task found 2023-08-25T04:13:43.969 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-25T04:13:43.987 INFO:teuthology.task.internal:Saving configuration 2023-08-25T04:13:44.010 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-25T04:13:44.020 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T04:13:44.047 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.945115', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T04:13:44.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.948839', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T04:13:44.084 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.947108', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T04:13:44.084 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-25T04:13:44.093 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-08-25T04:13:44.094 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-08-25T04:13:44.094 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-08-25T04:13:44.094 INFO:teuthology.run_tasks:Running task console_log... 2023-08-25T04:13:44.241 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4627b6b160>, signals=[15]) 2023-08-25T04:13:44.243 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-25T04:13:44.253 INFO:teuthology.task.internal:Opening connections... 2023-08-25T04:13:44.254 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:13:44.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:13:44.327 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2023-08-25T04:13:44.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:13:44.395 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2023-08-25T04:13:44.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:13:44.463 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-25T04:13:44.473 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2023-08-25T04:13:44.493 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2023-08-25T04:13:44.494 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-08-25T04:13:44.553 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T04:13:44.553 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T04:13:44.553 INFO:teuthology.orchestra.run.smithi145.stdout:ID="rhel" 2023-08-25T04:13:44.553 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="fedora" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8.6" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T04:13:44.554 INFO:teuthology.orchestra.run.smithi145.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T04:13:44.555 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T04:13:44.555 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:13:44.555 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T04:13:44.555 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T04:13:44.555 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T04:13:44.556 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T04:13:44.556 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-08-25T04:13:44.583 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2023-08-25T04:13:44.604 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-08-25T04:13:44.605 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-08-25T04:13:44.664 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:ID="rhel" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="fedora" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8.6" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2023-08-25T04:13:44.665 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T04:13:44.666 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2023-08-25T04:13:44.666 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T04:13:44.666 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T04:13:44.666 INFO:teuthology.orchestra.run.smithi162.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T04:13:44.667 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T04:13:44.668 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-08-25T04:13:44.692 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-08-25T04:13:44.714 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-08-25T04:13:44.714 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-08-25T04:13:44.773 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.6" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-08-25T04:13:44.774 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T04:13:44.775 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T04:13:44.776 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T04:13:44.776 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T04:13:44.776 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-08-25T04:13:44.799 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-25T04:13:44.811 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-25T04:13:44.822 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-25T04:13:44.823 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2023-08-25T04:13:44.825 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2023-08-25T04:13:44.827 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2023-08-25T04:13:44.845 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-25T04:13:44.855 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-25T04:13:44.856 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2023-08-25T04:13:44.884 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2023-08-25T04:13:44.886 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2023-08-25T04:13:44.918 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-25T04:13:44.994 INFO:teuthology.run_tasks:Running task kernel... 2023-08-25T04:13:45.015 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-25T04:13:45.016 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-08-25T04:13:45.017 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-08-25T04:13:45.017 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T04:13:45.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:13:45.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-08-25T04:13:45.245 DEBUG:teuthology.task.kernel:sha1 for client.0 is a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T04:13:45.245 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a6a69670761025e039dc5ae86619b0cf04c46dc2"... 2023-08-25T04:13:45.246 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2023-08-25T04:13:45.265 INFO:teuthology.orchestra.run.smithi145.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-25T04:13:45.265 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T04:13:45.266 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-25T04:13:45.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum upgrade -y linux-firmware 2023-08-25T04:13:46.508 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-08-25T04:13:46.508 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2023-08-25T04:13:47.994 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-08-25T04:13:48.645 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-08-25T04:13:52.128 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-08-25T04:13:52.832 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-08-25T04:13:52.833 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-08-25T04:13:52.833 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-08-25T04:13:52.962 INFO:teuthology.task.kernel:Downloading kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on client.0... 2023-08-25T04:13:52.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T04:13:53.180 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/ 2023-08-25T04:13:53.180 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/ --input-file=- 2023-08-25T04:13:53.183 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi145.front.sepia.ceph.com to complete... 2023-08-25T04:14:58.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25 04:14:58 URL:https://3.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm [380759896/380759896] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-25T04:14:58.388 INFO:teuthology.orchestra.run.smithi145.stderr:FINISHED --2023-08-25 04:14:58-- 2023-08-25T04:14:58.388 INFO:teuthology.orchestra.run.smithi145.stderr:Total wall clock time: 1m 5s 2023-08-25T04:14:58.389 INFO:teuthology.orchestra.run.smithi145.stderr:Downloaded: 1 files, 363M in 1m 5s (5.62 MB/s) 2023-08-25T04:14:58.425 INFO:teuthology.task.kernel:Installing kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on client.0... 2023-08-25T04:14:58.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-25T04:15:00.435 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying... ######################################## 2023-08-25T04:15:00.485 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing... ######################################## 2023-08-25T04:15:00.572 INFO:teuthology.orchestra.run.smithi145.stdout:Updating / installing... 2023-08-25T04:15:38.578 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-6.5.0_rc7_ga6a696707610-1 ######################################## 2023-08-25T04:16:05.673 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-25T04:16:05.673 DEBUG:teuthology.orchestra.run.smithi145:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-25T04:16:05.725 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T04:16:05.726 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/config-6.5.0-rc7-ga6a696707610 2023-08-25T04:16:05.726 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/vmlinuz-6.5.0-rc7-ga6a696707610 2023-08-25T04:16:05.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610 2023-08-25T04:16:05.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel 2023-08-25T04:16:05.726 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-25T04:16:05.727 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-25T04:16:05.728 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-25T04:16:05.729 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce 2023-08-25T04:16:05.730 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm.ko 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ansi_cprng.ko 2023-08-25T04:16:05.731 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/anubis.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/arc4.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_memcpy.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_pq.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_tx.ko 2023-08-25T04:16:05.732 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_xor.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/raid6test.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authenc.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authencesn.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blake2b_generic.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_common.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_generic.ko 2023-08-25T04:16:05.733 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/camellia_generic.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast5_generic.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast6_generic.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast_common.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ccm.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cmac.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crc32_generic.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_engine.ko 2023-08-25T04:16:05.734 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_user.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cts.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/des_generic.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/dh_generic.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/echainiv.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/essiv.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/fcrypt.ko 2023-08-25T04:16:05.735 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/khazad.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4hc.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/md4.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/michael_mic.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcbc.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcrypt.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/rmd160.ko 2023-08-25T04:16:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/seed.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/serpent_generic.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tcrypt.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tea.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_common.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_generic.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/vmac.ko 2023-08-25T04:16:05.737 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/wp512.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xcbc.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xor.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xxhash_generic.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_pad.ko 2023-08-25T04:16:05.738 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/custom_method.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/ec_sys.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/platform_profile.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbs.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbshc.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/video.ko 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata 2023-08-25T04:16:05.739 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/acard-ahci.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ahci_platform.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ata_generic.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/libahci_platform.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_acpi.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ali.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_amd.ko 2023-08-25T04:16:05.740 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_artop.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atiixp.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atp867x.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd640.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd64x.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cypress.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_efar.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt366.ko 2023-08-25T04:16:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt37x.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it8213.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it821x.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_jmicron.ko 2023-08-25T04:16:05.742 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_marvell.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_mpiix.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_netcell.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ninja32.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87410.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87415.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_oldpiix.ko 2023-08-25T04:16:05.743 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_opti.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_optidma.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pcmcia.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_piccolo.ko 2023-08-25T04:16:05.744 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_rdc.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sch.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_serverworks.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sil680.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sis.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sl82c105.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_triflex.ko 2023-08-25T04:16:05.745 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_via.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pdc_adma.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_inic162x.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_mv.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_nv.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_promise.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_qstor.ko 2023-08-25T04:16:05.746 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil.ko 2023-08-25T04:16:05.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil24.ko 2023-08-25T04:16:05.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sis.ko 2023-08-25T04:16:05.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_svw.ko 2023-08-25T04:16:05.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sx4.ko 2023-08-25T04:16:05.747 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_uli.ko 2023-08-25T04:16:05.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_via.ko 2023-08-25T04:16:05.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_vsc.ko 2023-08-25T04:16:05.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay 2023-08-25T04:16:05.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay/ks0108.ko 2023-08-25T04:16:05.748 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma/bcma.ko 2023-08-25T04:16:05.749 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe/aoe.ko 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/brd.ko 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd/drbd.ko 2023-08-25T04:16:05.750 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/floppy.ko 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/loop.ko 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/nbd.ko 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk 2023-08-25T04:16:05.751 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk/null_blk.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/pktcdvd.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/rbd.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/virtio_blk.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkfront.ko 2023-08-25T04:16:05.752 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram/zram.ko 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hangcheck-timer.ko 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-25T04:16:05.753 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/via-rng.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-25T04:16:05.754 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/lp.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave/mwave.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ppdev.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tlclk.ko 2023-08-25T04:16:05.755 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-25T04:16:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/uv_mmtimer.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/virtio_console.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-25T04:16:05.757 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp.ko 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat 2023-08-25T04:16:05.758 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-aes.ko 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-sha.ko 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-25T04:16:05.759 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca/dca.ko 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw 2023-08-25T04:16:05.760 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/amd64_edac.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/e752x_edac.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/edac_mce_amd.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3000_edac.ko 2023-08-25T04:16:05.761 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3200_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5100_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5400_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7300_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7core_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i82975x_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/ie31200_edac.ko 2023-08-25T04:16:05.762 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/sb_edac.ko 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/x38_edac.ko 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/edd.ko 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-ich.ko 2023-08-25T04:16:05.763 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-appleir.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-aureal.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-axff.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-dr.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elecom.ko 2023-08-25T04:16:05.764 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elo.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-emsff.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gaff.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gt683r.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gyration.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-25T04:16:05.765 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtekff.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-hyperv.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-icade.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-keytouch.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-kye.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lcpower.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-led.ko 2023-08-25T04:16:05.766 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lenovo.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-multitouch.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-ortek.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-penmount.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-petalynx.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-picolcd.ko 2023-08-25T04:16:05.767 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-pl.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-plantronics.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-primax.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-rmi.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-common.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-25T04:16:05.768 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-25T04:16:05.769 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-saitek.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-samsung.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sjoy.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sony.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-speedlink.ko 2023-08-25T04:16:05.770 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-steelseries.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sunplus.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tivo.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tmff.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-topseed.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-twinhan.ko 2023-08-25T04:16:05.771 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-uclogic.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-waltop.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-wiimote.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-xinmo.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zpff.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zydacron.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/uhid.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/wacom.ko 2023-08-25T04:16:05.772 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_balloon.ko 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_utils.ko 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_vmbus.ko 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru.ko 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru3.ko 2023-08-25T04:16:05.773 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7414.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7418.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adc128d818.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1025.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1026.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1029.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1031.ko 2023-08-25T04:16:05.774 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm9240.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ads7828.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7410.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7411.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7462.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7470.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7475.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7x10.ko 2023-08-25T04:16:05.775 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/amc6821.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/applesmc.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asb100.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asc7621.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/atxp1.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/coretemp.ko 2023-08-25T04:16:05.776 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dme1737.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds1621.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds620.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc1403.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc6w201.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71805f.ko 2023-08-25T04:16:05.777 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71882fg.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f75375s.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fam15h_power.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fschmd.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g760a.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g762.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl518sm.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl520sm.ko 2023-08-25T04:16:05.778 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5500_temp.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5k_amb.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmaem.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmpex.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina209.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina2xx.ko 2023-08-25T04:16:05.779 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/it87.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k10temp.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k8temp.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lineage-pem.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm63.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm73.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm75.ko 2023-08-25T04:16:05.780 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm77.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm78.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm80.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm83.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm85.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm87.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm90.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm92.ko 2023-08-25T04:16:05.781 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm93.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95234.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95241.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95245.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc2945.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4151.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4215.ko 2023-08-25T04:16:05.782 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4222.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4245.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4260.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4261.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max16065.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1619.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1668.ko 2023-08-25T04:16:05.783 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max197.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6639.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6650.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6697.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/mcp3021.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6683.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775-core.ko 2023-08-25T04:16:05.784 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct7802.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87360.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87427.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pcf8591.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-25T04:16:05.785 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-25T04:16:05.786 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/powr1220.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5627.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5636.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-25T04:16:05.787 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht15.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht21.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/shtc1.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sis5595.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47b397.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m1.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m192.ko 2023-08-25T04:16:05.788 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/thmc50.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp102.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp103.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp401.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp421.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via-cputemp.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via686a.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt1211.ko 2023-08-25T04:16:05.789 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt8231.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627ehf.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627hf.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83781d.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83791d.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83792d.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83793.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83795.ko 2023-08-25T04:16:05.790 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l785ts.ko 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l786ng.ko 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses 2023-08-25T04:16:05.791 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-25T04:16:05.792 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-25T04:16:05.793 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-25T04:16:05.794 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-dev.ko 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-mux.ko 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-smbus.ko 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-stub.ko 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core 2023-08-25T04:16:05.795 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_core.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw 2023-08-25T04:16:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-25T04:16:05.797 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-25T04:16:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-25T04:16:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/gameport.ko 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/lightning.ko 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/ns558.ko 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/joydev.ko 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse 2023-08-25T04:16:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/appletouch.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/bcm5974.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/cyapatp.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/sermouse.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-25T04:16:05.801 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/altera_ps2.ko 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/arc_ps2.ko 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/serio_raw.ko 2023-08-25T04:16:05.802 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/sparse-keymap.ko 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-blinkm.ko 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lm3530.ko 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lp3944.ko 2023-08-25T04:16:05.803 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-ss4200.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-25T04:16:05.804 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache/bcache.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-bio-prison.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache-smq.ko 2023-08-25T04:16:05.805 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-crypt.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-delay.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-flakey.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-log-userspace.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-multipath.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-queue-length.ko 2023-08-25T04:16:05.806 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-raid.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-round-robin.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-service-time.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-switch.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-thin-pool.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-verity.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/faulty.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/linear.ko 2023-08-25T04:16:05.807 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/multipath.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid0.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid1.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid10.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid456.ko 2023-08-25T04:16:05.808 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptbase.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptctl.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptfc.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptlan.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptsas.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptscsih.ko 2023-08-25T04:16:05.809 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptspi.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_ich.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_sch.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/mfd-core.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/sm501.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/viperboard.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/vx855.ko 2023-08-25T04:16:05.810 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/wl1273-core.ko 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds9802als.ko 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds990x.ko 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/bh1770glc.ko 2023-08-25T04:16:05.811 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710/cb710.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/at24.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/max6875.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/enclosure.ko 2023-08-25T04:16:05.812 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/hpilo.ko 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29003.ko 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29020.ko 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei 2023-08-25T04:16:05.813 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-me.ko 2023-08-25T04:16:05.814 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-txe.ko 2023-08-25T04:16:05.814 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei.ko 2023-08-25T04:16:05.815 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru 2023-08-25T04:16:05.815 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_7xx1.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_core.ko 2023-08-25T04:16:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tsl2550.ko 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_balloon.ko 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding/bonding.ko 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa 2023-08-25T04:16:05.817 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dummy.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/eql.ko 2023-08-25T04:16:05.818 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390 2023-08-25T04:16:05.819 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-25T04:16:05.820 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-25T04:16:05.821 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx 2023-08-25T04:16:05.822 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-25T04:16:05.823 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-25T04:16:05.824 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-25T04:16:05.825 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip 2023-08-25T04:16:05.826 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink 2023-08-25T04:16:05.827 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dnet.ko 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ethoc.ko 2023-08-25T04:16:05.828 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/fealnx.ko 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k 2023-08-25T04:16:05.829 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-25T04:16:05.830 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/jme.ko 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell 2023-08-25T04:16:05.831 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-25T04:16:05.832 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-25T04:16:05.833 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines 2023-08-25T04:16:05.834 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-25T04:16:05.835 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-25T04:16:05.836 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-25T04:16:05.837 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro 2023-08-25T04:16:05.838 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-25T04:16:05.839 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-25T04:16:05.840 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/geneve.ko 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv 2023-08-25T04:16:05.841 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ifb.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvlan.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvtap.ko 2023-08-25T04:16:05.842 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mii.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/net_failover.ko 2023-08-25T04:16:05.843 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/netconsole.ko 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/nlmon.ko 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/amd.ko 2023-08-25T04:16:05.844 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm87xx.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/broadcom.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/cicada.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/davicom.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/dp83640.ko 2023-08-25T04:16:05.845 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/et1011c.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/icplus.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/lxt.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/marvell.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/micrel.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/national.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/phylink.ko 2023-08-25T04:16:05.846 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/qsemi.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/realtek.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/smsc.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/ste10Xp.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/vitesse.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_async.ko 2023-08-25T04:16:05.847 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppoe.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppox.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pptp.ko 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip 2023-08-25T04:16:05.848 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slhc.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slip.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/sungem_phy.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tap.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-25T04:16:05.849 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_random.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tun.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/veth.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/virtio_net.ko 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3 2023-08-25T04:16:05.850 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan/vxlan.ko 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netfront.ko 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb/ntb.ko 2023-08-25T04:16:05.851 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme-core.ko 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme.ko 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport.ko 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_cs.ko 2023-08-25T04:16:05.852 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_pc.ko 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_serial.ko 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/xen-pcifront.ko 2023-08-25T04:16:05.853 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/i82092.ko 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/pd6729.ko 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform 2023-08-25T04:16:05.854 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acerhdf.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-25T04:16:05.855 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/hdaps.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-25T04:16:05.856 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel_ips.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-25T04:16:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi.ko 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients 2023-08-25T04:16:05.858 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps_parport.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/pps_core.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp_kvm.ko 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc 2023-08-25T04:16:05.859 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-25T04:16:05.860 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-em3027.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-25T04:16:05.861 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-max6900.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-25T04:16:05.862 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-25T04:16:05.863 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-x1205.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-9xxx.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-sas.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-xxxx.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/BusLogic.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/a100u2w.ko 2023-08-25T04:16:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/advansys.ko 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/am53c974.ko 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr 2023-08-25T04:16:05.865 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/atp870u.ko 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa/bfa.ko 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc 2023-08-25T04:16:05.866 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ch.ko 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-25T04:16:05.867 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dc395x.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-25T04:16:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dmx3191d.ko 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esp_scsi.ko 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe 2023-08-25T04:16:05.869 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic/fnic.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hpsa.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hptiop.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hv_storvsc.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/initio.ko 2023-08-25T04:16:05.870 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ipr.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ips.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci/isci.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc/libfc.ko 2023-08-25T04:16:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi.ko 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas/libsas.ko 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid 2023-08-25T04:16:05.872 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-25T04:16:05.873 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvumi.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pmcraid.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla1280.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-25T04:16:05.874 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/raid_class.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_debug.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-25T04:16:05.875 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ses.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/st.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/stex.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/virtio_scsi.ko 2023-08-25T04:16:05.876 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/wd719x.ko 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb/ssb.ko 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback 2023-08-25T04:16:05.877 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_file.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_iblock.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_mod.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_pscsi.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_user.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-25T04:16:05.878 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-25T04:16:05.879 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-25T04:16:05.880 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_gsm.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_hdlc.ko 2023-08-25T04:16:05.881 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/nozomi.ko 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/arc_uart.ko 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-25T04:16:05.882 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/synclink_gt.ko 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio.ko 2023-08-25T04:16:05.883 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_aec.ko 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_cif.ko 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_pci_generic.ko 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_sercos3.ko 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-acm.ko 2023-08-25T04:16:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usblp.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usbtmc.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/mdc800.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/microtek.ko 2023-08-25T04:16:05.885 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/adutux.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/appledisplay.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi26.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi62.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ezusb.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/idmouse.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/iowarrior.ko 2023-08-25T04:16:05.886 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ldusb.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/legousbtower.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usb3503.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usblcd.ko 2023-08-25T04:16:05.887 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/uss720.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/yurex.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy/phy-generic.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/aircable.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ark3116.ko 2023-08-25T04:16:05.888 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ch341.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cp210x.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cyberjack.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/empeg.ko 2023-08-25T04:16:05.889 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_ti.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipaq.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipw.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ir-usb.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-25T04:16:05.890 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mct_u232.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7720.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7840.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/navman.ko 2023-08-25T04:16:05.891 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/omninet.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/opticon.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/option.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/oti6858.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/pl2303.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcaux.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcserial.ko 2023-08-25T04:16:05.892 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/quatech2.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/safe_serial.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/sierra.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ssu100.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/symbolserial.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-25T04:16:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_debug.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/visor.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/whiteheat.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/uas.ko 2023-08-25T04:16:05.894 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-karma.ko 2023-08-25T04:16:05.895 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/usb-storage.ko 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip 2023-08-25T04:16:05.896 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio.ko 2023-08-25T04:16:05.897 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost.ko 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_net.ko 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_scsi.ko 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight 2023-08-25T04:16:05.898 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/apple_bl.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/lcd.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_balloon.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_mmio.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci.ko 2023-08-25T04:16:05.899 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_ring.ko 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2482.ko 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2490.ko 2023-08-25T04:16:05.900 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-25T04:16:05.901 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/wire.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-25T04:16:05.902 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/hpwdt.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/i6300esb.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ib700wdt.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ibmasr.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-25T04:16:05.903 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it87_wdt.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/machzwd.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/nv_tco.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-25T04:16:05.904 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/softdog.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/via_wdt.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/wdt_pci.ko 2023-08-25T04:16:05.905 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/xen_wdt.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-evtchn.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntalloc.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntdev.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-25T04:16:05.906 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-privcmd.ko 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p/9p.ko 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs/affs.ko 2023-08-25T04:16:05.907 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs/befs.ko 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/binfmt_misc.ko 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs/btrfs.ko 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles/cachefiles.ko 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph 2023-08-25T04:16:05.908 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph/ceph.ko 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda/coda.ko 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs/cramfs.ko 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm/dlm.ko 2023-08-25T04:16:05.909 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs/f2fs.ko 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/fat.ko 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/msdos.ko 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/vfat.ko 2023-08-25T04:16:05.910 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache/fscache.ko 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/cuse.ko 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/fuse.ko 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2/gfs2.ko 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs 2023-08-25T04:16:05.911 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs/hfs.ko 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus/hfsplus.ko 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs/isofs.ko 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs/jfs.ko 2023-08-25T04:16:05.912 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd/lockd.ko 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix/minix.ko 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs/netfs.ko 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs 2023-08-25T04:16:05.913 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfs.ko 2023-08-25T04:16:05.914 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv3.ko 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv4.ko 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/grace.ko 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/nfs_acl.ko 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd/nfsd.ko 2023-08-25T04:16:05.915 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2/nilfs2.ko 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-celtic.ko 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-centeuro.ko 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-croatian.ko 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-cyrillic.ko 2023-08-25T04:16:05.916 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-gaelic.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-greek.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-iceland.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-inuit.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-roman.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-romanian.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-turkish.ko 2023-08-25T04:16:05.917 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1250.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1251.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1255.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp737.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp775.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp850.ko 2023-08-25T04:16:05.918 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp852.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp855.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp857.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp860.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp861.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp862.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp863.ko 2023-08-25T04:16:05.919 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp864.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp865.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp866.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp869.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp874.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp932.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp936.ko 2023-08-25T04:16:05.920 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp949.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp950.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_euc-jp.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-1.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-13.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-14.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-15.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-2.ko 2023-08-25T04:16:05.921 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-3.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-4.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-5.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-6.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-7.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-9.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-r.ko 2023-08-25T04:16:05.922 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-ru.ko 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-u.ko 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_utf8.ko 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-25T04:16:05.923 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2.ko 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs 2023-08-25T04:16:05.924 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs/overlay.ko 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore/ramoops.ko 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs/reiserfs.ko 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs/romfs.ko 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb 2023-08-25T04:16:05.925 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client/cifs.ko 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_arc4.ko 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_md4.ko 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs/squashfs.ko 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv 2023-08-25T04:16:05.926 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv/sysv.ko 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf/udf.ko 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs/ufs.ko 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs 2023-08-25T04:16:05.927 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs/xfs.ko 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu/rcutorture.ko 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/torture.ko 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib 2023-08-25T04:16:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/asn1_encoder.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc-itu-t.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc8.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libarc4.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libdes.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/libcrc32c.ko 2023-08-25T04:16:05.929 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lru_cache.ko 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4_compress.ko 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4hc_compress.ko 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math/cordic.ko 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6 2023-08-25T04:16:05.930 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6/raid6_pq.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_bm.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_fsm.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_kmp.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd/zstd_compress.ko 2023-08-25T04:16:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm/hwpoison-inject.ko 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/6lowpan.ko 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_dest.ko 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_fragment.ko 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_hop.ko 2023-08-25T04:16:05.932 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_mobility.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_routing.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_udp.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/garp.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/mrp.ko 2023-08-25T04:16:05.933 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/p8022.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/psnap.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/stp.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q/8021q.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_fd.ko 2023-08-25T04:16:05.934 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_rdma.ko 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_virtio.ko 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv/batman-adv.ko 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/br_netfilter.ko 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/bridge.ko 2023-08-25T04:16:05.935 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-25T04:16:05.936 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-25T04:16:05.937 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-25T04:16:05.938 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtables.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph/libceph.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/failover.ko 2023-08-25T04:16:05.939 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/pktgen.ko 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp.ko 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_diag.ko 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv4.ko 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv6.ko 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver 2023-08-25T04:16:05.940 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver/dns_resolver.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/dsa_core.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_brcm.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_dsa.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_none.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_trailer.ko 2023-08-25T04:16:05.941 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154.ko 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ah4.ko 2023-08-25T04:16:05.942 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/esp4.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/fou.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/gre.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/inet_diag.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_gre.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_tunnel.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_vti.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipcomp.ko 2023-08-25T04:16:05.943 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipip.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-25T04:16:05.944 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-25T04:16:05.945 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-25T04:16:05.946 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_bic.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_dctcp.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_diag.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_highspeed.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_htcp.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_hybla.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_illinois.ko 2023-08-25T04:16:05.947 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_lp.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_scalable.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_vegas.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_veno.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_westwood.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_yeah.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tunnel4.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_diag.ko 2023-08-25T04:16:05.948 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_tunnel.ko 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ah6.ko 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/esp6.ko 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/fou6.ko 2023-08-25T04:16:05.949 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_tunnel.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_vti.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ipcomp6.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-25T04:16:05.950 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-25T04:16:05.951 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-25T04:16:05.952 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/sit.ko 2023-08-25T04:16:05.953 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/tunnel6.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key/af_key.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_core.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_eth.ko 2023-08-25T04:16:05.954 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip.ko 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip6.ko 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_netlink.ko 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ppp.ko 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc/llc.ko 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154 2023-08-25T04:16:05.955 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154/mac802154.ko 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls/mpls_gso.ko 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set.ko 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-25T04:16:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-25T04:16:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-25T04:16:05.958 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-25T04:16:05.959 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conncount.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack.ko 2023-08-25T04:16:05.960 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-25T04:16:05.961 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_log_syslog.ko 2023-08-25T04:16:05.962 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_irc.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_sip.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_tables.ko 2023-08-25T04:16:05.963 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_log.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_chain_nat.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_compat.ko 2023-08-25T04:16:05.964 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_ct.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fib.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_hash.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_limit.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_log.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_masq.ko 2023-08-25T04:16:05.965 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_nat.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_numgen.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_queue.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_quota.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_redir.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject_inet.ko 2023-08-25T04:16:05.966 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_socket.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_tunnel.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_AUDIT.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CT.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_DSCP.ko 2023-08-25T04:16:05.967 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HL.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HMARK.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LED.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LOG.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NETMAP.ko 2023-08-25T04:16:05.968 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFLOG.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_RATEEST.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_SECMARK.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TEE.ko 2023-08-25T04:16:05.969 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TPROXY.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TRACE.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_addrtype.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_bpf.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cgroup.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cluster.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_comment.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connbytes.ko 2023-08-25T04:16:05.970 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlabel.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlimit.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connmark.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_conntrack.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cpu.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dccp.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_devgroup.ko 2023-08-25T04:16:05.971 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dscp.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ecn.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_esp.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hashlimit.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_helper.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hl.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipcomp.ko 2023-08-25T04:16:05.972 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_iprange.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipvs.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_l2tp.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_length.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_limit.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mac.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mark.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_multiport.ko 2023-08-25T04:16:05.973 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nat.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nfacct.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_osf.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_owner.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_physdev.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_pkttype.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_policy.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_quota.ko 2023-08-25T04:16:05.974 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_rateest.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_realm.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_recent.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_sctp.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_set.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_socket.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_state.ko 2023-08-25T04:16:05.975 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_statistic.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_string.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_tcpmss.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_time.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_u32.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink/netlink_diag.ko 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh 2023-08-25T04:16:05.976 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh/nsh.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/openvswitch.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-geneve.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-gre.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-vxlan.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet/af_packet_diag.ko 2023-08-25T04:16:05.977 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds.ko 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_rdma.ko 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_tcp.ko 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_csum.ko 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_gact.ko 2023-08-25T04:16:05.978 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_ipt.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_mirred.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_nat.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_pedit.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_police.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_simple.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_skbedit.ko 2023-08-25T04:16:05.979 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_vlan.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_basic.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_bpf.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_flow.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_fw.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_route.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_u32.ko 2023-08-25T04:16:05.980 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_cmp.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_ipset.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_meta.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_nbyte.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_text.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_u32.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_choke.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_codel.ko 2023-08-25T04:16:05.981 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_drr.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_fq.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_gred.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hfsc.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hhf.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_htb.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_ingress.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio.ko 2023-08-25T04:16:05.982 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio_lib.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_multiq.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_netem.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_pie.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_plug.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_prio.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_qfq.ko 2023-08-25T04:16:05.983 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_red.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfb.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfq.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_tbf.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_teql.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp_diag.ko 2023-08-25T04:16:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/sunrpc.ko 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-25T04:16:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/diag.ko 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/tipc.ko 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix/unix_diag.ko 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-25T04:16:05.986 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock.ko 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys 2023-08-25T04:16:05.987 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys/trusted.ko 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib/irqbypass.ko 2023-08-25T04:16:05.988 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias.bin 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.bin 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.modinfo 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep.bin 2023-08-25T04:16:05.989 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.devname 2023-08-25T04:16:05.990 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.order 2023-08-25T04:16:05.990 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.softdep 2023-08-25T04:16:05.990 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols 2023-08-25T04:16:05.990 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols.bin 2023-08-25T04:16:05.991 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-ga6a696707610 2023-08-25T04:16:05.991 DEBUG:teuthology.orchestra.run.smithi145:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-25T04:16:06.019 INFO:teuthology.orchestra.run.smithi145.stdout:postinstall scriptlet (using /bin/sh): 2023-08-25T04:16:06.019 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-ga6a696707610 -a -r /boot/System.map-6.5.0-rc7-ga6a696707610 ]; then 2023-08-25T04:16:06.019 INFO:teuthology.orchestra.run.smithi145.stdout:cp /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T04:16:06.019 INFO:teuthology.orchestra.run.smithi145.stdout:cp /boot/System.map-6.5.0-rc7-ga6a696707610 /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/installkernel 6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-25T04:16:06.020 INFO:teuthology.orchestra.run.smithi145.stdout:new-kernel-pkg --remove 6.5.0-rc7-ga6a696707610 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-ga6a696707610.img 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-install remove 6.5.0-rc7-ga6a696707610 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-ga6a696707610 2023-08-25T04:16:06.021 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2023-08-25T04:16:06.022 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2023-08-25T04:16:06.102 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2023-08-25T04:16:06.102 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2023-08-25T04:16:06.102 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.02 2023-08-25T04:16:06.103 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 123.el8 2023-08-25T04:16:06.103 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2023-08-25T04:16:06.103 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-08-25T04:16:06.103 INFO:teuthology.orchestra.run.smithi145.stdout:Group : System Environment/Base 2023-08-25T04:16:06.104 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 9265450 2023-08-25T04:16:06.104 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2023-08-25T04:16:06.104 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-08-25T04:16:06.105 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-08-25T04:16:06.106 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-08-25T04:16:06.106 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-08-25T04:16:06.106 INFO:teuthology.orchestra.run.smithi145.stdout:Relocations : (not relocatable) 2023-08-25T04:16:06.106 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : Red Hat, Inc. 2023-08-25T04:16:06.107 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : Red Hat, Inc. 2023-08-25T04:16:06.107 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-25T04:16:06.107 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2023-08-25T04:16:06.108 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2023-08-25T04:16:06.108 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:16:06.108 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-25T04:16:06.108 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-25T04:16:06.109 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-25T04:16:06.109 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2023-08-25T04:16:06.109 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:16:06.109 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2023-08-25T04:16:06.111 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-25T04:16:06.111 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 6.5.0-rc7-ga6a696707610 2023-08-25T04:16:06.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-25T04:16:08.516 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2023-08-25T04:16:10.394 INFO:teuthology.orchestra.run.smithi145.stderr:done 2023-08-25T04:16:10.454 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2023-08-25T04:16:10.472 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.MuwxkDQNxv 2023-08-25T04:16:10.473 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MuwxkDQNxv 2023-08-25T04:16:10.546 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.MuwxkDQNxv 2023-08-25T04:16:10.694 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.MuwxkDQNxv is 6KB 2023-08-25T04:16:10.715 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.MuwxkDQNxv 2023-08-25T04:16:10.730 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2023-08-25T04:16:10.810 INFO:teuthology.orchestra.run.smithi145.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-08-25T04:16:10.811 INFO:teuthology.orchestra.run.smithi145.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-08-25T04:16:10.811 INFO:teuthology.orchestra.run.smithi145.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-ga6a696707610.conf 2023-08-25T04:16:10.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-ga6a696707610 2023-08-25T04:16:10.953 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2023-08-25T04:16:40.957 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-08-25T04:16:40.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-08-25T04:16:40.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:17:40.960 DEBUG:teuthology.orchestra.remote:timed out 2023-08-25T04:17:49.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-08-25T04:17:49.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:17:57.471 DEBUG:teuthology.orchestra.run.smithi145:> true 2023-08-25T04:17:57.993 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-08-25T04:17:57.993 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-ga6a696707610"... 2023-08-25T04:17:57.994 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2023-08-25T04:17:58.036 INFO:teuthology.orchestra.run.smithi145.stdout:6.5.0-rc7-ga6a696707610 2023-08-25T04:17:58.037 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-ga6a696707610 vs 6.5.0-rc7-ga6a696707610 2023-08-25T04:17:58.037 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-25T04:17:58.037 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-08-25T04:17:59.039 DEBUG:teuthology.parallel:result is None 2023-08-25T04:17:59.040 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-25T04:17:59.047 INFO:teuthology.task.internal:Creating test directory... 2023-08-25T04:17:59.048 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T04:17:59.052 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T04:17:59.055 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T04:17:59.090 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-25T04:17:59.097 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-25T04:17:59.107 INFO:teuthology.task.internal:Creating archive directory... 2023-08-25T04:17:59.107 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T04:17:59.135 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T04:17:59.137 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T04:17:59.192 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-25T04:17:59.198 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-25T04:17:59.199 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T04:17:59.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:17:59.220 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T04:17:59.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:17:59.238 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T04:17:59.242 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T04:17:59.262 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T04:17:59.307 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.316 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.327 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.331 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.374 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.418 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T04:17:59.421 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-25T04:17:59.430 INFO:teuthology.task.internal:Configuring sudo... 2023-08-25T04:17:59.430 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T04:17:59.434 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T04:17:59.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T04:17:59.499 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-25T04:17:59.508 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-25T04:17:59.509 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T04:17:59.513 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T04:17:59.515 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T04:17:59.550 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:17:59.662 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:17:59.745 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:17:59.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:17:59.898 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:17:59.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T04:17:59.981 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:18:00.027 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:18:00.086 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:18:00.171 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:18:00.218 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:18:00.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T04:18:00.288 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:18:00.331 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T04:18:00.400 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:18:00.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T04:18:00.489 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:18:00.490 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T04:18:00.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2023-08-25T04:18:00.562 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2023-08-25T04:18:00.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2023-08-25T04:18:00.647 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T04:18:00.653 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T04:18:00.673 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T04:18:01.148 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-25T04:18:01.156 INFO:teuthology.task.internal:Starting timer... 2023-08-25T04:18:01.157 INFO:teuthology.run_tasks:Running task pcp... 2023-08-25T04:18:01.167 INFO:teuthology.run_tasks:Running task selinux... 2023-08-25T04:18:01.176 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-08-25T04:18:01.176 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2023-08-25T04:18:01.306 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2023-08-25T04:18:01.310 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2023-08-25T04:18:01.367 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2023-08-25T04:18:01.369 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2023-08-25T04:18:01.447 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2023-08-25T04:18:01.448 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-25T04:18:01.448 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2023-08-25T04:18:01.507 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2023-08-25T04:18:01.508 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2023-08-25T04:18:01.541 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2023-08-25T04:18:01.542 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2023-08-25T04:18:01.567 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2023-08-25T04:18:01.567 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi145.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2023-08-25T04:18:01.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T04:18:01.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:18:01.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T04:18:01.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:18:01.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T04:18:01.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:18:01.693 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-25T04:18:01.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2023-08-25T04:18:01.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2023-08-25T04:18:01.787 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2023-08-25T04:18:01.815 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-25T04:18:01.825 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-25T04:18:01.826 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-25T04:18:01.848 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-25T04:18:01.849 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi145.front.sepia.ceph.com,smithi162.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-25T04:31:36.541 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi145.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2023-08-25T04:31:36.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-08-25T04:31:36.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:31:36.710 DEBUG:teuthology.orchestra.run.smithi145:> true 2023-08-25T04:31:36.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-08-25T04:31:36.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-08-25T04:31:36.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:31:36.896 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-08-25T04:31:36.967 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-08-25T04:31:36.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-08-25T04:31:36.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T04:31:37.039 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-08-25T04:31:37.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-08-25T04:31:37.111 INFO:teuthology.run_tasks:Running task clock... 2023-08-25T04:31:37.127 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-25T04:31:37.128 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T04:31:37.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T04:31:37.134 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T04:31:37.134 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T04:31:37.139 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T04:31:37.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T04:31:37.173 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T04:31:37.188 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T04:31:37.189 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T04:31:37.204 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T04:31:37.210 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T04:31:37.246 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T04:31:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2023-08-25T04:31:37.276 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2023-08-25T04:31:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2023-08-25T04:31:37.291 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T04:31:37.297 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2023-08-25T04:31:37.306 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T04:31:37.312 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T04:31:37.326 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T04:31:37.337 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2023-08-25T04:31:37.367 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2023-08-25T04:31:37.389 INFO:teuthology.orchestra.run.smithi162.stderr:bash: ntpq: command not found 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.393 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.394 INFO:teuthology.orchestra.run.smithi183.stderr:bash: ntpq: command not found 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.397 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.399 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T04:31:37.434 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T04:31:37.541 INFO:teuthology.orchestra.run.smithi145.stderr:bash: ntpq: command not found 2023-08-25T04:31:37.542 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T04:31:37.542 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2023-08-25T04:31:37.543 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.543 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.543 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.543 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T04:31:37.544 INFO:teuthology.run_tasks:Running task pexec... 2023-08-25T04:31:37.556 INFO:teuthology.task.pexec:Executing custom commands... 2023-08-25T04:31:37.557 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:31:37.558 DEBUG:teuthology.orchestra.run.smithi145:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T04:31:37.558 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi162.front.sepia.ceph.com 2023-08-25T04:31:37.559 DEBUG:teuthology.orchestra.run.smithi162:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T04:31:37.559 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi183.front.sepia.ceph.com 2023-08-25T04:31:37.559 DEBUG:teuthology.orchestra.run.smithi183:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-25T04:31:37.932 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:37.933 INFO:teuthology.orchestra.run.smithi162.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:38.040 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:38.814 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:05:07 ago on Fri 25 Aug 2023 04:26:31 AM UTC. 2023-08-25T04:31:38.827 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:04:56 ago on Fri 25 Aug 2023 04:26:42 AM UTC. 2023-08-25T04:31:39.048 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:00 ago on Fri 25 Aug 2023 04:29:38 AM UTC. 2023-08-25T04:31:40.566 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-08-25T04:31:40.567 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-08-25T04:31:40.568 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2023-08-25T04:31:40.568 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-08-25T04:31:40.568 INFO:teuthology.orchestra.run.smithi183.stdout:Resetting modules: 2023-08-25T04:31:40.569 INFO:teuthology.orchestra.run.smithi183.stdout: container-tools 2023-08-25T04:31:40.569 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:40.569 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-08-25T04:31:40.570 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2023-08-25T04:31:40.570 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:40.574 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-08-25T04:31:40.574 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout:Resetting modules: 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout: container-tools 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:40.575 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-08-25T04:31:40.576 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-08-25T04:31:40.576 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:40.849 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-08-25T04:31:40.850 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-08-25T04:31:40.851 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2023-08-25T04:31:40.852 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-08-25T04:31:40.852 INFO:teuthology.orchestra.run.smithi145.stdout:Resetting modules: 2023-08-25T04:31:40.852 INFO:teuthology.orchestra.run.smithi145.stdout: container-tools 2023-08-25T04:31:40.853 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:40.853 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-08-25T04:31:40.853 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-08-25T04:31:40.853 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:41.559 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:05:10 ago on Fri 25 Aug 2023 04:26:31 AM UTC. 2023-08-25T04:31:41.603 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:04:59 ago on Fri 25 Aug 2023 04:26:42 AM UTC. 2023-08-25T04:31:42.089 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:03 ago on Fri 25 Aug 2023 04:29:38 AM UTC. 2023-08-25T04:31:44.889 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-08-25T04:31:45.026 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-08-25T04:31:45.159 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-08-25T04:31:45.364 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:45.505 INFO:teuthology.orchestra.run.smithi162.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:45.754 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-08-25T04:31:46.239 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:05:14 ago on Fri 25 Aug 2023 04:26:31 AM UTC. 2023-08-25T04:31:46.379 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:05:03 ago on Fri 25 Aug 2023 04:26:42 AM UTC. 2023-08-25T04:31:46.800 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:08 ago on Fri 25 Aug 2023 04:29:38 AM UTC. 2023-08-25T04:31:48.016 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-08-25T04:31:48.102 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-08-25T04:31:48.110 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================================ 2023-08-25T04:31:48.110 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================================ 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout:Installing group/module packages: 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T04:31:48.111 INFO:teuthology.orchestra.run.smithi183.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout:Installing module profiles: 2023-08-25T04:31:48.112 INFO:teuthology.orchestra.run.smithi183.stdout: container-tools/common 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout:Enabling module streams: 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout: container-tools rhel8 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================================ 2023-08-25T04:31:48.113 INFO:teuthology.orchestra.run.smithi183.stdout:Install 10 Packages 2023-08-25T04:31:48.114 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:48.114 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 18 M 2023-08-25T04:31:48.114 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 65 M 2023-08-25T04:31:48.115 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2023-08-25T04:31:48.195 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================================ 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================================ 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout:Installing group/module packages: 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T04:31:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T04:31:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T04:31:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T04:31:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T04:31:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T04:31:48.197 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout:Installing module profiles: 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: container-tools/common 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout:Enabling module streams: 2023-08-25T04:31:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: container-tools rhel8 2023-08-25T04:31:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:48.199 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-08-25T04:31:48.199 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================================ 2023-08-25T04:31:48.199 INFO:teuthology.orchestra.run.smithi162.stdout:Install 10 Packages 2023-08-25T04:31:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:48.200 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 18 M 2023-08-25T04:31:48.200 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 65 M 2023-08-25T04:31:48.200 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-08-25T04:31:48.225 INFO:teuthology.orchestra.run.smithi183.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 221 kB/s | 25 kB 00:00 2023-08-25T04:31:48.242 INFO:teuthology.orchestra.run.smithi183.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 725 kB/s | 94 kB 00:00 2023-08-25T04:31:48.268 INFO:teuthology.orchestra.run.smithi183.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 264 kB/s | 41 kB 00:00 2023-08-25T04:31:48.361 INFO:teuthology.orchestra.run.smithi162.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 575 kB/s | 94 kB 00:00 2023-08-25T04:31:48.387 INFO:teuthology.orchestra.run.smithi162.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 216 kB/s | 41 kB 00:00 2023-08-25T04:31:48.401 INFO:teuthology.orchestra.run.smithi183.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-08-25T04:31:48.445 INFO:teuthology.orchestra.run.smithi162.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 100 kB/s | 25 kB 00:00 2023-08-25T04:31:48.485 INFO:teuthology.orchestra.run.smithi183.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 190 kB/s | 49 kB 00:00 2023-08-25T04:31:48.513 INFO:teuthology.orchestra.run.smithi162.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.1 MB/s | 524 kB 00:00 2023-08-25T04:31:48.519 INFO:teuthology.orchestra.run.smithi183.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-08-25T04:31:48.546 INFO:teuthology.orchestra.run.smithi162.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 268 kB/s | 49 kB 00:00 2023-08-25T04:31:48.561 INFO:teuthology.orchestra.run.smithi183.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-08-25T04:31:48.571 INFO:teuthology.orchestra.run.smithi162.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-08-25T04:31:48.587 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-08-25T04:31:48.621 INFO:teuthology.orchestra.run.smithi162.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-08-25T04:31:48.686 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================================ 2023-08-25T04:31:48.686 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-08-25T04:31:48.686 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================================ 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout:Installing group/module packages: 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-25T04:31:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout:Installing module profiles: 2023-08-25T04:31:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: container-tools/common 2023-08-25T04:31:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:48.689 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling module streams: 2023-08-25T04:31:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: container-tools rhel8 2023-08-25T04:31:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:48.689 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-08-25T04:31:48.690 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================================ 2023-08-25T04:31:48.690 INFO:teuthology.orchestra.run.smithi145.stdout:Install 10 Packages 2023-08-25T04:31:48.690 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:48.690 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 18 M 2023-08-25T04:31:48.691 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 65 M 2023-08-25T04:31:48.691 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-08-25T04:31:48.728 INFO:teuthology.orchestra.run.smithi183.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 17 MB/s | 8.0 MB 00:00 2023-08-25T04:31:48.738 INFO:teuthology.orchestra.run.smithi162.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-08-25T04:31:48.764 INFO:teuthology.orchestra.run.smithi145.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-08-25T04:31:48.789 INFO:teuthology.orchestra.run.smithi145.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 248 kB/s | 25 kB 00:00 2023-08-25T04:31:48.795 INFO:teuthology.orchestra.run.smithi183.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.4 MB/s | 2.2 MB 00:00 2023-08-25T04:31:48.814 INFO:teuthology.orchestra.run.smithi145.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 325 kB/s | 41 kB 00:00 2023-08-25T04:31:48.840 INFO:teuthology.orchestra.run.smithi145.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 649 kB/s | 49 kB 00:00 2023-08-25T04:31:48.873 INFO:teuthology.orchestra.run.smithi145.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-08-25T04:31:48.895 INFO:teuthology.orchestra.run.smithi162.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-08-25T04:31:48.899 INFO:teuthology.orchestra.run.smithi145.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-08-25T04:31:48.912 INFO:teuthology.orchestra.run.smithi183.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-08-25T04:31:48.912 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:31:48.912 INFO:teuthology.orchestra.run.smithi183.stdout:Total 23 MB/s | 18 MB 00:00 2023-08-25T04:31:48.958 INFO:teuthology.orchestra.run.smithi145.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.5 MB/s | 213 kB 00:00 2023-08-25T04:31:48.989 INFO:teuthology.orchestra.run.smithi162.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-08-25T04:31:48.989 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:31:48.989 INFO:teuthology.orchestra.run.smithi162.stdout:Total 23 MB/s | 18 MB 00:00 2023-08-25T04:31:49.083 INFO:teuthology.orchestra.run.smithi145.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-08-25T04:31:49.112 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2023-08-25T04:31:49.131 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2023-08-25T04:31:49.131 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2023-08-25T04:31:49.192 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-08-25T04:31:49.211 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-08-25T04:31:49.211 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-08-25T04:31:49.225 INFO:teuthology.orchestra.run.smithi145.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-08-25T04:31:49.313 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2023-08-25T04:31:49.317 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2023-08-25T04:31:49.341 INFO:teuthology.orchestra.run.smithi145.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-08-25T04:31:49.342 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:31:49.342 INFO:teuthology.orchestra.run.smithi145.stdout:Total 28 MB/s | 18 MB 00:00 2023-08-25T04:31:49.394 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-08-25T04:31:49.398 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-08-25T04:31:49.673 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-08-25T04:31:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-08-25T04:31:49.696 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-08-25T04:31:49.816 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2023-08-25T04:31:49.968 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-08-25T04:31:50.015 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T04:31:50.190 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T04:31:50.196 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:50.206 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-08-25T04:31:50.211 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-08-25T04:31:50.384 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T04:31:50.399 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:50.602 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T04:31:50.618 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T04:31:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-08-25T04:31:50.828 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T04:31:50.991 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T04:31:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-25T04:31:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:51.218 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T04:31:51.441 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-25T04:31:51.715 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-25T04:31:51.760 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T04:31:52.032 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T04:31:52.087 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-25T04:31:52.653 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T04:31:52.888 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T04:31:52.913 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-25T04:31:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T04:31:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T04:31:53.181 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T04:31:53.263 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:53.389 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T04:31:53.525 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:53.525 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T04:31:53.525 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T04:31:53.526 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T04:31:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-25T04:31:53.925 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:54.156 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-25T04:31:54.229 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T04:31:54.230 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T04:31:54.231 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T04:31:54.231 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T04:31:54.370 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-25T04:31:54.555 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:54.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T04:31:54.665 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:05:23 ago on Fri 25 Aug 2023 04:26:31 AM UTC. 2023-08-25T04:31:54.946 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-25T04:31:54.946 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-25T04:31:54.947 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-25T04:31:54.948 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-25T04:31:55.445 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T04:31:55.446 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:05:12 ago on Fri 25 Aug 2023 04:26:42 AM UTC. 2023-08-25T04:31:56.497 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-25T04:31:56.497 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:17 ago on Fri 25 Aug 2023 04:29:38 AM UTC. 2023-08-25T04:31:57.720 INFO:teuthology.orchestra.run.smithi183.stdout:Installed products updated. 2023-08-25T04:31:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:57.917 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2023-08-25T04:31:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:57.917 INFO:teuthology.orchestra.run.smithi183.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T04:31:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T04:31:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T04:31:57.918 INFO:teuthology.orchestra.run.smithi183.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:57.919 INFO:teuthology.orchestra.run.smithi183.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:57.919 INFO:teuthology.orchestra.run.smithi183.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:57.919 INFO:teuthology.orchestra.run.smithi183.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T04:31:57.920 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:31:57.920 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-08-25T04:31:58.078 DEBUG:teuthology.parallel:result is None 2023-08-25T04:31:59.294 INFO:teuthology.orchestra.run.smithi162.stdout:Installed products updated. 2023-08-25T04:31:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T04:31:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:31:59.506 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-08-25T04:31:59.699 DEBUG:teuthology.parallel:result is None 2023-08-25T04:31:59.727 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-08-25T04:31:59.937 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:59.937 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-08-25T04:31:59.937 INFO:teuthology.orchestra.run.smithi145.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.937 INFO:teuthology.orchestra.run.smithi145.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-25T04:31:59.938 INFO:teuthology.orchestra.run.smithi145.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-25T04:31:59.939 INFO:teuthology.orchestra.run.smithi145.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-25T04:31:59.939 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:31:59.939 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-08-25T04:32:00.163 DEBUG:teuthology.parallel:result is None 2023-08-25T04:32:00.164 INFO:teuthology.run_tasks:Running task install... 2023-08-25T04:32:00.172 DEBUG:teuthology.task.install:project ceph 2023-08-25T04:32:00.172 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}} 2023-08-25T04:32:00.173 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'} 2023-08-25T04:32:00.173 INFO:teuthology.task.install:Using flavor: default 2023-08-25T04:32:00.179 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-25T04:32:00.179 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-25T04:32:00.180 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T04:32:00.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:32:00.182 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T04:32:00.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:32:00.184 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T04:32:00.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:32:00.330 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T04:32:00.331 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T04:32:00.400 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T04:32:00.400 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T04:32:00.409 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T04:32:00.410 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T04:32:00.509 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T04:32:00.510 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:32:00.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T04:32:00.575 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T04:32:00.576 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:32:00.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T04:32:00.578 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T04:32:00.578 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T04:32:00.588 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T04:32:00.588 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:32:00.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T04:32:00.604 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T04:32:00.605 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T04:32:00.615 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T04:32:00.616 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T04:32:00.671 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T04:32:00.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T04:32:00.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T04:32:00.750 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T04:32:00.758 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T04:32:00.784 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T04:32:00.818 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2023-08-25T04:32:00.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2023-08-25T04:32:00.825 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2023-08-25T04:32:00.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2023-08-25T04:32:00.853 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2023-08-25T04:32:00.856 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2023-08-25T04:32:01.226 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:01.232 INFO:teuthology.orchestra.run.smithi162.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:01.304 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:01.619 INFO:teuthology.orchestra.run.smithi162.stdout:44 files removed 2023-08-25T04:32:01.632 INFO:teuthology.orchestra.run.smithi183.stdout:44 files removed 2023-08-25T04:32:01.673 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T04:32:01.689 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T04:32:01.741 INFO:teuthology.orchestra.run.smithi145.stdout:44 files removed 2023-08-25T04:32:01.793 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T04:32:02.036 INFO:teuthology.orchestra.run.smithi162.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:02.071 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:02.235 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-08-25T04:32:02.744 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 270 kB/s | 81 kB 00:00 2023-08-25T04:32:02.868 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 221 kB/s | 81 kB 00:00 2023-08-25T04:32:03.024 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 228 kB/s | 81 kB 00:00 2023-08-25T04:32:03.104 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-08-25T04:32:03.252 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-08-25T04:32:03.345 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-08-25T04:32:03.413 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-08-25T04:32:03.495 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-08-25T04:32:03.614 INFO:teuthology.orchestra.run.smithi162.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-08-25T04:32:03.621 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-08-25T04:32:03.638 INFO:teuthology.orchestra.run.smithi183.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-08-25T04:32:03.798 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-08-25T04:32:04.272 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2023-08-25T04:32:04.536 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-08-25T04:32:05.741 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 16 MB 00:01 2023-08-25T04:32:08.020 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 422 kB/s | 24 kB 00:00 2023-08-25T04:32:08.452 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 183 kB/s | 24 kB 00:00 2023-08-25T04:32:08.925 INFO:teuthology.orchestra.run.smithi183.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-08-25T04:32:09.553 INFO:teuthology.orchestra.run.smithi162.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-08-25T04:32:09.915 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 246 kB/s | 24 kB 00:00 2023-08-25T04:32:11.082 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-08-25T04:32:17.119 INFO:teuthology.orchestra.run.smithi183.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-08-25T04:32:17.790 INFO:teuthology.orchestra.run.smithi162.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-08-25T04:32:20.291 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 MB/s | 47 MB 00:01 2023-08-25T04:32:28.226 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:28.226 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:28.241 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.242 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.243 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T04:32:28.244 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T04:32:28.246 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T04:32:28.250 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.251 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T04:32:28.383 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-08-25T04:32:28.395 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-08-25T04:32:28.396 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2023-08-25T04:32:28.396 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-08-25T04:32:28.396 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2023-08-25T04:32:28.397 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T04:32:28.397 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T04:32:28.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T04:32:28.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T04:32:28.398 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T04:32:28.398 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T04:32:28.398 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T04:32:28.398 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T04:32:28.399 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T04:32:28.399 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T04:32:28.399 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T04:32:28.399 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T04:32:28.400 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T04:32:28.400 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T04:32:28.400 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T04:32:28.400 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T04:32:28.401 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T04:32:28.401 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T04:32:28.401 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T04:32:28.401 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T04:32:28.402 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T04:32:28.402 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T04:32:28.402 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:28.402 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T04:32:28.403 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T04:32:28.403 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T04:32:28.403 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T04:32:28.403 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T04:32:28.404 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T04:32:28.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T04:32:28.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T04:32:28.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T04:32:28.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T04:32:28.405 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T04:32:28.405 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T04:32:28.406 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T04:32:28.406 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T04:32:28.406 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2023-08-25T04:32:28.406 INFO:teuthology.orchestra.run.smithi183.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T04:32:28.407 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T04:32:28.407 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T04:32:28.407 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2023-08-25T04:32:28.407 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T04:32:28.408 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T04:32:28.408 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T04:32:28.408 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T04:32:28.408 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T04:32:28.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T04:32:28.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T04:32:28.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T04:32:28.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T04:32:28.410 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T04:32:28.410 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T04:32:28.410 INFO:teuthology.orchestra.run.smithi183.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T04:32:28.410 INFO:teuthology.orchestra.run.smithi183.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T04:32:28.410 INFO:teuthology.orchestra.run.smithi183.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T04:32:28.411 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T04:32:28.412 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T04:32:28.413 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T04:32:28.413 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T04:32:28.413 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T04:32:28.413 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T04:32:28.413 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T04:32:28.414 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T04:32:28.415 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T04:32:28.416 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T04:32:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T04:32:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T04:32:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T04:32:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T04:32:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T04:32:28.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T04:32:28.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T04:32:28.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T04:32:28.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T04:32:28.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T04:32:28.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T04:32:28.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T04:32:28.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T04:32:28.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T04:32:28.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T04:32:28.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T04:32:28.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:28.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T04:32:28.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T04:32:28.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T04:32:28.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T04:32:28.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T04:32:28.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T04:32:28.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T04:32:28.423 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout:Enabling module streams: 2023-08-25T04:32:28.424 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql 10 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout:Install 128 Packages 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 3 Packages 2023-08-25T04:32:28.425 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:32:28.426 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 206 M 2023-08-25T04:32:28.426 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2023-08-25T04:32:28.569 INFO:teuthology.orchestra.run.smithi183.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 51 kB/s | 6.4 kB 00:00 2023-08-25T04:32:28.770 INFO:teuthology.orchestra.run.smithi183.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 4.2 MB/s | 862 kB 00:00 2023-08-25T04:32:28.828 INFO:teuthology.orchestra.run.smithi183.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 2.8 MB/s | 167 kB 00:00 2023-08-25T04:32:28.888 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:28.889 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:28.904 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.905 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.907 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T04:32:28.908 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T04:32:28.910 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T04:32:28.914 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T04:32:28.915 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T04:32:29.050 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-08-25T04:32:29.062 INFO:teuthology.orchestra.run.smithi162.stdout:=========================================================================================================================== 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout:=========================================================================================================================== 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T04:32:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T04:32:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T04:32:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:29.066 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T04:32:29.067 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T04:32:29.068 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T04:32:29.069 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T04:32:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T04:32:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T04:32:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T04:32:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T04:32:29.071 INFO:teuthology.orchestra.run.smithi162.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T04:32:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T04:32:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T04:32:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T04:32:29.072 INFO:teuthology.orchestra.run.smithi162.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T04:32:29.073 INFO:teuthology.orchestra.run.smithi162.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T04:32:29.074 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T04:32:29.075 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T04:32:29.076 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T04:32:29.077 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T04:32:29.078 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T04:32:29.079 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T04:32:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T04:32:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T04:32:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T04:32:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T04:32:29.080 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T04:32:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T04:32:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T04:32:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T04:32:29.081 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T04:32:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T04:32:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T04:32:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T04:32:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T04:32:29.082 INFO:teuthology.orchestra.run.smithi162.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T04:32:29.083 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T04:32:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T04:32:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T04:32:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T04:32:29.084 INFO:teuthology.orchestra.run.smithi162.stdout:Enabling module streams: 2023-08-25T04:32:29.084 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql 10 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout:=========================================================================================================================== 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout:Install 128 Packages 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 3 Packages 2023-08-25T04:32:29.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:32:29.086 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 206 M 2023-08-25T04:32:29.086 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-08-25T04:32:29.145 INFO:teuthology.orchestra.run.smithi183.stdout:(4/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 7.2 MB/s | 2.3 MB 00:00 2023-08-25T04:32:29.362 INFO:teuthology.orchestra.run.smithi183.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 7.2 MB/s | 1.6 MB 00:00 2023-08-25T04:32:29.365 INFO:teuthology.orchestra.run.smithi162.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-08-25T04:32:29.621 INFO:teuthology.orchestra.run.smithi183.stdout:(6/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 4.6 MB/s | 5.4 MB 00:01 2023-08-25T04:32:30.063 INFO:teuthology.orchestra.run.smithi183.stdout:(7/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 6.6 MB/s | 4.6 MB 00:00 2023-08-25T04:32:30.455 INFO:teuthology.orchestra.run.smithi183.stdout:(8/131): ceph-common-18.0.0-5703.gba6b39e4.el8. 11 MB/s | 22 MB 00:02 2023-08-25T04:32:30.474 INFO:teuthology.orchestra.run.smithi162.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 778 kB/s | 862 kB 00:01 2023-08-25T04:32:30.497 INFO:teuthology.orchestra.run.smithi183.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 587 kB/s | 24 kB 00:00 2023-08-25T04:32:30.624 INFO:teuthology.orchestra.run.smithi162.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 1.1 MB/s | 167 kB 00:00 2023-08-25T04:32:31.425 INFO:teuthology.orchestra.run.smithi162.stdout:(4/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-08-25T04:32:31.816 INFO:teuthology.orchestra.run.smithi162.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 4.0 MB/s | 1.6 MB 00:00 2023-08-25T04:32:32.030 INFO:teuthology.orchestra.run.smithi183.stdout:(10/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el 6.9 MB/s | 14 MB 00:01 2023-08-25T04:32:32.094 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:32.095 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T04:32:32.111 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:32.112 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T04:32:32.113 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T04:32:32.114 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T04:32:32.117 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T04:32:32.120 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T04:32:32.121 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T04:32:32.122 INFO:teuthology.orchestra.run.smithi183.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 2.8 MB/s | 260 kB 00:00 2023-08-25T04:32:32.206 INFO:teuthology.orchestra.run.smithi183.stdout:(12/131): libcephfs-devel-18.0.0-5703.gba6b39e4 373 kB/s | 31 kB 00:00 2023-08-25T04:32:32.263 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-08-25T04:32:32.275 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2023-08-25T04:32:32.275 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-08-25T04:32:32.276 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2023-08-25T04:32:32.276 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-08-25T04:32:32.276 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T04:32:32.276 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T04:32:32.277 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T04:32:32.277 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T04:32:32.278 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T04:32:32.279 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T04:32:32.279 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T04:32:32.279 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T04:32:32.280 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T04:32:32.280 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T04:32:32.280 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T04:32:32.280 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T04:32:32.280 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T04:32:32.281 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T04:32:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T04:32:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T04:32:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T04:32:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T04:32:32.282 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T04:32:32.283 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T04:32:32.284 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T04:32:32.285 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T04:32:32.285 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T04:32:32.285 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T04:32:32.285 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T04:32:32.285 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T04:32:32.286 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T04:32:32.287 INFO:teuthology.orchestra.run.smithi145.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T04:32:32.287 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T04:32:32.287 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T04:32:32.287 INFO:teuthology.orchestra.run.smithi145.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T04:32:32.288 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T04:32:32.288 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T04:32:32.288 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T04:32:32.288 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T04:32:32.288 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T04:32:32.289 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T04:32:32.290 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T04:32:32.290 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T04:32:32.290 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T04:32:32.290 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T04:32:32.291 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T04:32:32.292 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T04:32:32.292 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T04:32:32.292 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T04:32:32.293 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T04:32:32.293 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T04:32:32.293 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T04:32:32.294 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T04:32:32.294 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T04:32:32.294 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T04:32:32.295 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T04:32:32.295 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T04:32:32.295 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T04:32:32.296 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T04:32:32.296 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T04:32:32.296 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T04:32:32.297 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T04:32:32.297 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T04:32:32.297 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:32.297 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T04:32:32.298 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T04:32:32.298 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T04:32:32.298 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T04:32:32.298 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T04:32:32.299 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T04:32:32.299 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T04:32:32.299 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T04:32:32.299 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T04:32:32.299 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T04:32:32.300 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T04:32:32.301 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T04:32:32.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling module streams: 2023-08-25T04:32:32.303 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql 10 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:Install 128 Packages 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 3 Packages 2023-08-25T04:32:32.304 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:32:32.305 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 206 M 2023-08-25T04:32:32.305 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-08-25T04:32:32.367 INFO:teuthology.orchestra.run.smithi162.stdout:(6/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 1.3 MB/s | 2.3 MB 00:01 2023-08-25T04:32:32.406 INFO:teuthology.orchestra.run.smithi183.stdout:(13/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 3.8 MB/s | 784 kB 00:00 2023-08-25T04:32:32.434 INFO:teuthology.orchestra.run.smithi162.stdout:(7/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 7.5 MB/s | 4.6 MB 00:00 2023-08-25T04:32:32.473 INFO:teuthology.orchestra.run.smithi183.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 2.0 MB/s | 139 kB 00:00 2023-08-25T04:32:32.573 INFO:teuthology.orchestra.run.smithi183.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 1.4 MB/s | 139 kB 00:00 2023-08-25T04:32:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-08-25T04:32:32.807 INFO:teuthology.orchestra.run.smithi183.stdout:(16/131): libradosstriper1-18.0.0-5703.gba6b39e 2.0 MB/s | 473 kB 00:00 2023-08-25T04:32:33.165 INFO:teuthology.orchestra.run.smithi183.stdout:(17/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x8 5.0 MB/s | 18 MB 00:03 2023-08-25T04:32:33.257 INFO:teuthology.orchestra.run.smithi183.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 488 kB/s | 45 kB 00:00 2023-08-25T04:32:33.366 INFO:teuthology.orchestra.run.smithi183.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 1.1 MB/s | 126 kB 00:00 2023-08-25T04:32:33.459 INFO:teuthology.orchestra.run.smithi145.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.0 MB/s | 862 kB 00:00 2023-08-25T04:32:33.474 INFO:teuthology.orchestra.run.smithi183.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 1.8 MB/s | 197 kB 00:00 2023-08-25T04:32:33.591 INFO:teuthology.orchestra.run.smithi183.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 3.1 MB/s | 371 kB 00:00 2023-08-25T04:32:33.659 INFO:teuthology.orchestra.run.smithi145.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 836 kB/s | 167 kB 00:00 2023-08-25T04:32:33.708 INFO:teuthology.orchestra.run.smithi183.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 3.0 MB/s | 364 kB 00:00 2023-08-25T04:32:33.817 INFO:teuthology.orchestra.run.smithi183.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 983 kB/s | 107 kB 00:00 2023-08-25T04:32:33.909 INFO:teuthology.orchestra.run.smithi183.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.0 MB/s | 94 kB 00:00 2023-08-25T04:32:34.000 INFO:teuthology.orchestra.run.smithi162.stdout:(8/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el8 8.7 MB/s | 14 MB 00:01 2023-08-25T04:32:34.092 INFO:teuthology.orchestra.run.smithi162.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 266 kB/s | 24 kB 00:00 2023-08-25T04:32:34.409 INFO:teuthology.orchestra.run.smithi183.stdout:(25/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 4.2 MB/s | 6.8 MB 00:01 2023-08-25T04:32:34.501 INFO:teuthology.orchestra.run.smithi183.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 1.9 MB/s | 179 kB 00:00 2023-08-25T04:32:34.593 INFO:teuthology.orchestra.run.smithi183.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 252 kB/s | 23 kB 00:00 2023-08-25T04:32:34.693 INFO:teuthology.orchestra.run.smithi183.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 1.3 MB/s | 132 kB 00:00 2023-08-25T04:32:34.752 INFO:teuthology.orchestra.run.smithi183.stdout:(29/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 4.2 MB/s | 3.5 MB 00:00 2023-08-25T04:32:34.768 INFO:teuthology.orchestra.run.smithi145.stdout:(4/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-08-25T04:32:35.138 INFO:teuthology.orchestra.run.smithi183.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 4.0 MB/s | 1.7 MB 00:00 2023-08-25T04:32:35.223 INFO:teuthology.orchestra.run.smithi183.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 2.8 MB/s | 242 kB 00:00 2023-08-25T04:32:35.319 INFO:teuthology.orchestra.run.smithi183.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 474 kB/s | 47 kB 00:00 2023-08-25T04:32:35.394 INFO:teuthology.orchestra.run.smithi183.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 193 kB/s | 15 kB 00:00 2023-08-25T04:32:35.470 INFO:teuthology.orchestra.run.smithi183.stdout:(34/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 3.4 MB/s | 261 kB 00:00 2023-08-25T04:32:35.553 INFO:teuthology.orchestra.run.smithi183.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 2.0 MB/s | 171 kB 00:00 2023-08-25T04:32:35.562 INFO:teuthology.orchestra.run.smithi145.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 2.0 MB/s | 1.6 MB 00:00 2023-08-25T04:32:35.629 INFO:teuthology.orchestra.run.smithi183.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 239 kB/s | 18 kB 00:00 2023-08-25T04:32:36.363 INFO:teuthology.orchestra.run.smithi183.stdout:(37/131): ceph-test-18.0.0-5703.gba6b39e4.el8.x 8.2 MB/s | 48 MB 00:05 2023-08-25T04:32:36.388 INFO:teuthology.orchestra.run.smithi183.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 403 kB/s | 306 kB 00:00 2023-08-25T04:32:36.488 INFO:teuthology.orchestra.run.smithi183.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-08-25T04:32:36.522 INFO:teuthology.orchestra.run.smithi183.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-08-25T04:32:36.597 INFO:teuthology.orchestra.run.smithi183.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 583 kB/s | 44 kB 00:00 2023-08-25T04:32:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-08-25T04:32:36.748 INFO:teuthology.orchestra.run.smithi183.stdout:(43/131): liboath-2.6.2-3.el8.x86_64.rpm 153 kB/s | 59 kB 00:00 2023-08-25T04:32:36.773 INFO:teuthology.orchestra.run.smithi183.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 438 kB/s | 33 kB 00:00 2023-08-25T04:32:36.823 INFO:teuthology.orchestra.run.smithi183.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-08-25T04:32:36.865 INFO:teuthology.orchestra.run.smithi183.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-08-25T04:32:36.869 INFO:teuthology.orchestra.run.smithi145.stdout:(6/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 3.5 MB/s | 4.6 MB 00:01 2023-08-25T04:32:36.899 INFO:teuthology.orchestra.run.smithi183.stdout:(47/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-08-25T04:32:36.941 INFO:teuthology.orchestra.run.smithi183.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-08-25T04:32:36.966 INFO:teuthology.orchestra.run.smithi183.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-08-25T04:32:37.008 INFO:teuthology.orchestra.run.smithi183.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-08-25T04:32:37.033 INFO:teuthology.orchestra.run.smithi183.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-08-25T04:32:37.084 INFO:teuthology.orchestra.run.smithi183.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 648 kB/s | 49 kB 00:00 2023-08-25T04:32:37.109 INFO:teuthology.orchestra.run.smithi183.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 779 kB/s | 59 kB 00:00 2023-08-25T04:32:37.111 INFO:teuthology.orchestra.run.smithi145.stdout:(7/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 1.1 MB/s | 5.4 MB 00:04 2023-08-25T04:32:37.142 INFO:teuthology.orchestra.run.smithi183.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-08-25T04:32:37.168 INFO:teuthology.orchestra.run.smithi183.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 720 kB/s | 60 kB 00:00 2023-08-25T04:32:37.193 INFO:teuthology.orchestra.run.smithi183.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-08-25T04:32:37.210 INFO:teuthology.orchestra.run.smithi183.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 232 kB/s | 16 kB 00:00 2023-08-25T04:32:37.235 INFO:teuthology.orchestra.run.smithi183.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 501 kB/s | 34 kB 00:00 2023-08-25T04:32:37.269 INFO:teuthology.orchestra.run.smithi183.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-08-25T04:32:37.286 INFO:teuthology.orchestra.run.smithi183.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 849 kB/s | 64 kB 00:00 2023-08-25T04:32:37.303 INFO:teuthology.orchestra.run.smithi183.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 278 kB/s | 19 kB 00:00 2023-08-25T04:32:37.328 INFO:teuthology.orchestra.run.smithi183.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 406 kB/s | 24 kB 00:00 2023-08-25T04:32:37.345 INFO:teuthology.orchestra.run.smithi183.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 498 kB/s | 29 kB 00:00 2023-08-25T04:32:37.362 INFO:teuthology.orchestra.run.smithi183.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-08-25T04:32:37.406 INFO:teuthology.orchestra.run.smithi183.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.2 MB/s | 255 kB 00:00 2023-08-25T04:32:37.431 INFO:teuthology.orchestra.run.smithi183.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-08-25T04:32:37.456 INFO:teuthology.orchestra.run.smithi183.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 649 kB/s | 61 kB 00:00 2023-08-25T04:32:37.482 INFO:teuthology.orchestra.run.smithi183.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-08-25T04:32:37.507 INFO:teuthology.orchestra.run.smithi183.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-08-25T04:32:37.532 INFO:teuthology.orchestra.run.smithi183.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-08-25T04:32:37.591 INFO:teuthology.orchestra.run.smithi183.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-08-25T04:32:37.624 INFO:teuthology.orchestra.run.smithi183.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-08-25T04:32:37.666 INFO:teuthology.orchestra.run.smithi183.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-08-25T04:32:37.700 INFO:teuthology.orchestra.run.smithi183.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 7.1 MB/s | 1.7 MB 00:00 2023-08-25T04:32:37.755 INFO:teuthology.orchestra.run.smithi183.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.6 MB/s | 323 kB 00:00 2023-08-25T04:32:37.800 INFO:teuthology.orchestra.run.smithi183.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2023-08-25T04:32:37.834 INFO:teuthology.orchestra.run.smithi183.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 9.1 MB/s | 688 kB 00:00 2023-08-25T04:32:37.868 INFO:teuthology.orchestra.run.smithi183.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-08-25T04:32:37.893 INFO:teuthology.orchestra.run.smithi183.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-08-25T04:32:37.918 INFO:teuthology.orchestra.run.smithi183.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-08-25T04:32:37.960 INFO:teuthology.orchestra.run.smithi183.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-08-25T04:32:37.994 INFO:teuthology.orchestra.run.smithi183.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 795 kB/s | 27 kB 00:00 2023-08-25T04:32:38.036 INFO:teuthology.orchestra.run.smithi183.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-25T04:32:38.078 INFO:teuthology.orchestra.run.smithi183.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-08-25T04:32:38.161 INFO:teuthology.orchestra.run.smithi183.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-08-25T04:32:38.303 INFO:teuthology.orchestra.run.smithi183.stdout:(86/131): python3-scipy-1.0.0-21.module+el8.5.0 17 MB/s | 14 MB 00:00 2023-08-25T04:32:38.337 INFO:teuthology.orchestra.run.smithi183.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.6 MB/s | 457 kB 00:00 2023-08-25T04:32:38.362 INFO:teuthology.orchestra.run.smithi183.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-08-25T04:32:38.387 INFO:teuthology.orchestra.run.smithi183.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-08-25T04:32:38.413 INFO:teuthology.orchestra.run.smithi183.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-25T04:32:38.438 INFO:teuthology.orchestra.run.smithi183.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-08-25T04:32:38.455 INFO:teuthology.orchestra.run.smithi183.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-08-25T04:32:38.488 INFO:teuthology.orchestra.run.smithi183.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-08-25T04:32:38.547 INFO:teuthology.orchestra.run.smithi183.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 8.0 MB/s | 3.7 MB 00:00 2023-08-25T04:32:38.567 INFO:teuthology.orchestra.run.smithi162.stdout:(10/131): ceph-test-18.0.0-5703.gba6b39e4.el8.x 11 MB/s | 48 MB 00:04 2023-08-25T04:32:38.572 INFO:teuthology.orchestra.run.smithi183.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-08-25T04:32:38.623 INFO:teuthology.orchestra.run.smithi183.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-08-25T04:32:38.625 INFO:teuthology.orchestra.run.smithi162.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 4.4 MB/s | 260 kB 00:00 2023-08-25T04:32:38.665 INFO:teuthology.orchestra.run.smithi183.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-08-25T04:32:38.667 INFO:teuthology.orchestra.run.smithi162.stdout:(12/131): libcephfs-devel-18.0.0-5703.gba6b39e4 748 kB/s | 31 kB 00:00 2023-08-25T04:32:38.740 INFO:teuthology.orchestra.run.smithi183.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-08-25T04:32:38.759 INFO:teuthology.orchestra.run.smithi162.stdout:(13/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 8.4 MB/s | 784 kB 00:00 2023-08-25T04:32:38.810 INFO:teuthology.orchestra.run.smithi162.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 2.7 MB/s | 139 kB 00:00 2023-08-25T04:32:38.849 INFO:teuthology.orchestra.run.smithi183.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-08-25T04:32:38.852 INFO:teuthology.orchestra.run.smithi162.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 3.3 MB/s | 139 kB 00:00 2023-08-25T04:32:38.890 INFO:teuthology.orchestra.run.smithi183.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-08-25T04:32:38.933 INFO:teuthology.orchestra.run.smithi183.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-08-25T04:32:38.935 INFO:teuthology.orchestra.run.smithi162.stdout:(16/131): libradosstriper1-18.0.0-5703.gba6b39e 5.6 MB/s | 473 kB 00:00 2023-08-25T04:32:38.966 INFO:teuthology.orchestra.run.smithi183.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-08-25T04:32:38.991 INFO:teuthology.orchestra.run.smithi183.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-08-25T04:32:39.016 INFO:teuthology.orchestra.run.smithi183.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-08-25T04:32:39.042 INFO:teuthology.orchestra.run.smithi183.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-25T04:32:39.067 INFO:teuthology.orchestra.run.smithi183.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-08-25T04:32:39.092 INFO:teuthology.orchestra.run.smithi183.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-25T04:32:39.117 INFO:teuthology.orchestra.run.smithi183.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 848 kB/s | 43 kB 00:00 2023-08-25T04:32:39.143 INFO:teuthology.orchestra.run.smithi183.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-08-25T04:32:39.160 INFO:teuthology.orchestra.run.smithi183.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-08-25T04:32:39.193 INFO:teuthology.orchestra.run.smithi183.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-08-25T04:32:39.219 INFO:teuthology.orchestra.run.smithi183.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-08-25T04:32:39.236 INFO:teuthology.orchestra.run.smithi183.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-08-25T04:32:39.311 INFO:teuthology.orchestra.run.smithi183.stdout:(114/131): postgresql-server-10.21-2.module+el8 7.8 MB/s | 5.1 MB 00:00 2023-08-25T04:32:39.336 INFO:teuthology.orchestra.run.smithi183.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 178 kB/s | 18 kB 00:00 2023-08-25T04:32:39.370 INFO:teuthology.orchestra.run.smithi183.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-08-25T04:32:39.395 INFO:teuthology.orchestra.run.smithi183.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.0 MB/s | 605 kB 00:00 2023-08-25T04:32:39.420 INFO:teuthology.orchestra.run.smithi183.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-08-25T04:32:39.445 INFO:teuthology.orchestra.run.smithi183.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-08-25T04:32:39.471 INFO:teuthology.orchestra.run.smithi183.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-08-25T04:32:39.496 INFO:teuthology.orchestra.run.smithi183.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-08-25T04:32:39.521 INFO:teuthology.orchestra.run.smithi183.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-25T04:32:39.544 INFO:teuthology.orchestra.run.smithi162.stdout:(17/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 11 MB/s | 6.8 MB 00:00 2023-08-25T04:32:39.547 INFO:teuthology.orchestra.run.smithi183.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-25T04:32:39.572 INFO:teuthology.orchestra.run.smithi183.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-08-25T04:32:39.594 INFO:teuthology.orchestra.run.smithi162.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 905 kB/s | 45 kB 00:00 2023-08-25T04:32:39.622 INFO:teuthology.orchestra.run.smithi183.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.5 MB/s | 644 kB 00:00 2023-08-25T04:32:39.648 INFO:teuthology.orchestra.run.smithi183.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-08-25T04:32:39.667 INFO:teuthology.orchestra.run.smithi162.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 1.7 MB/s | 126 kB 00:00 2023-08-25T04:32:39.673 INFO:teuthology.orchestra.run.smithi183.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-08-25T04:32:39.720 INFO:teuthology.orchestra.run.smithi162.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 3.7 MB/s | 197 kB 00:00 2023-08-25T04:32:39.723 INFO:teuthology.orchestra.run.smithi183.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-08-25T04:32:39.774 INFO:teuthology.orchestra.run.smithi183.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-08-25T04:32:39.778 INFO:teuthology.orchestra.run.smithi162.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 6.2 MB/s | 371 kB 00:00 2023-08-25T04:32:39.896 INFO:teuthology.orchestra.run.smithi162.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 3.1 MB/s | 364 kB 00:00 2023-08-25T04:32:39.962 INFO:teuthology.orchestra.run.smithi162.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 1.6 MB/s | 107 kB 00:00 2023-08-25T04:32:40.079 INFO:teuthology.orchestra.run.smithi162.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 803 kB/s | 94 kB 00:00 2023-08-25T04:32:40.482 INFO:teuthology.orchestra.run.smithi183.stdout:(130/131): librados2-18.0.0-5703.gba6b39e4.el8. 4.4 MB/s | 3.7 MB 00:00 2023-08-25T04:32:40.536 INFO:teuthology.orchestra.run.smithi145.stdout:(8/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el8 4.0 MB/s | 14 MB 00:03 2023-08-25T04:32:40.554 INFO:teuthology.orchestra.run.smithi162.stdout:(25/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 7.4 MB/s | 3.5 MB 00:00 2023-08-25T04:32:40.557 INFO:teuthology.orchestra.run.smithi183.stdout:(131/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 4.1 MB/s | 3.6 MB 00:00 2023-08-25T04:32:40.562 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:32:40.562 INFO:teuthology.orchestra.run.smithi183.stdout:Total 17 MB/s | 206 MB 00:12 2023-08-25T04:32:40.596 INFO:teuthology.orchestra.run.smithi162.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 4.2 MB/s | 179 kB 00:00 2023-08-25T04:32:40.655 INFO:teuthology.orchestra.run.smithi162.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 395 kB/s | 23 kB 00:00 2023-08-25T04:32:40.705 INFO:teuthology.orchestra.run.smithi162.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.6 MB/s | 132 kB 00:00 2023-08-25T04:32:40.745 INFO:teuthology.orchestra.run.smithi145.stdout:(9/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x86 4.6 MB/s | 18 MB 00:03 2023-08-25T04:32:40.770 INFO:teuthology.orchestra.run.smithi145.stdout:(10/131): ceph-selinux-18.0.0-5703.gba6b39e4.el 104 kB/s | 24 kB 00:00 2023-08-25T04:32:40.872 INFO:teuthology.orchestra.run.smithi162.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 11 MB/s | 1.7 MB 00:00 2023-08-25T04:32:40.879 INFO:teuthology.orchestra.run.smithi145.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 2.3 MB/s | 260 kB 00:00 2023-08-25T04:32:41.137 INFO:teuthology.orchestra.run.smithi145.stdout:(12/131): ceph-common-18.0.0-5703.gba6b39e4.el8 2.5 MB/s | 22 MB 00:08 2023-08-25T04:32:41.162 INFO:teuthology.orchestra.run.smithi145.stdout:(13/131): libcephfs-devel-18.0.0-5703.gba6b39e4 110 kB/s | 31 kB 00:00 2023-08-25T04:32:41.263 INFO:teuthology.orchestra.run.smithi145.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 1.4 MB/s | 139 kB 00:00 2023-08-25T04:32:41.363 INFO:teuthology.orchestra.run.smithi145.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 1.4 MB/s | 139 kB 00:00 2023-08-25T04:32:41.397 INFO:teuthology.orchestra.run.smithi145.stdout:(16/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 3.0 MB/s | 784 kB 00:00 2023-08-25T04:32:41.414 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2023-08-25T04:32:41.492 INFO:teuthology.orchestra.run.smithi145.stdout:(17/131): libradosstriper1-18.0.0-5703.gba6b39e 3.6 MB/s | 473 kB 00:00 2023-08-25T04:32:41.534 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2023-08-25T04:32:41.534 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2023-08-25T04:32:41.539 INFO:teuthology.orchestra.run.smithi162.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-08-25T04:32:41.547 INFO:teuthology.orchestra.run.smithi145.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 824 kB/s | 45 kB 00:00 2023-08-25T04:32:41.614 INFO:teuthology.orchestra.run.smithi145.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 1.8 MB/s | 126 kB 00:00 2023-08-25T04:32:41.628 INFO:teuthology.orchestra.run.smithi162.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 2.7 MB/s | 242 kB 00:00 2023-08-25T04:32:41.699 INFO:teuthology.orchestra.run.smithi145.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 2.3 MB/s | 197 kB 00:00 2023-08-25T04:32:41.798 INFO:teuthology.orchestra.run.smithi145.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 3.6 MB/s | 371 kB 00:00 2023-08-25T04:32:41.873 INFO:teuthology.orchestra.run.smithi162.stdout:(32/131): ceph-common-18.0.0-5703.gba6b39e4.el8 1.7 MB/s | 22 MB 00:12 2023-08-25T04:32:41.907 INFO:teuthology.orchestra.run.smithi145.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 3.3 MB/s | 364 kB 00:00 2023-08-25T04:32:41.907 INFO:teuthology.orchestra.run.smithi162.stdout:(33/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 170 kB/s | 47 kB 00:00 2023-08-25T04:32:41.932 INFO:teuthology.orchestra.run.smithi162.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5703.gb 248 kB/s | 15 kB 00:00 2023-08-25T04:32:41.957 INFO:teuthology.orchestra.run.smithi145.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 2.1 MB/s | 107 kB 00:00 2023-08-25T04:32:42.007 INFO:teuthology.orchestra.run.smithi145.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.8 MB/s | 94 kB 00:00 2023-08-25T04:32:42.141 INFO:teuthology.orchestra.run.smithi162.stdout:(35/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x8 1.8 MB/s | 18 MB 00:09 2023-08-25T04:32:42.166 INFO:teuthology.orchestra.run.smithi162.stdout:(36/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 733 kB/s | 171 kB 00:00 2023-08-25T04:32:42.194 INFO:teuthology.orchestra.run.smithi162.stdout:(37/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 918 kB/s | 261 kB 00:00 2023-08-25T04:32:42.217 INFO:teuthology.orchestra.run.smithi162.stdout:(38/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 239 kB/s | 18 kB 00:00 2023-08-25T04:32:42.383 INFO:teuthology.orchestra.run.smithi145.stdout:(25/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 6.9 MB/s | 6.8 MB 00:00 2023-08-25T04:32:42.433 INFO:teuthology.orchestra.run.smithi145.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 3.5 MB/s | 179 kB 00:00 2023-08-25T04:32:42.475 INFO:teuthology.orchestra.run.smithi145.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 558 kB/s | 23 kB 00:00 2023-08-25T04:32:42.525 INFO:teuthology.orchestra.run.smithi145.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.6 MB/s | 132 kB 00:00 2023-08-25T04:32:42.646 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2023-08-25T04:32:42.650 INFO:teuthology.orchestra.run.smithi162.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 128 kB/s | 59 kB 00:00 2023-08-25T04:32:42.651 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2023-08-25T04:32:42.667 INFO:teuthology.orchestra.run.smithi145.stdout:(29/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 5.4 MB/s | 3.5 MB 00:00 2023-08-25T04:32:42.718 INFO:teuthology.orchestra.run.smithi162.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 150 kB/s | 75 kB 00:00 2023-08-25T04:32:42.809 INFO:teuthology.orchestra.run.smithi145.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 6.2 MB/s | 1.7 MB 00:00 2023-08-25T04:32:42.876 INFO:teuthology.orchestra.run.smithi145.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 3.6 MB/s | 242 kB 00:00 2023-08-25T04:32:42.884 INFO:teuthology.orchestra.run.smithi162.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 427 kB/s | 306 kB 00:00 2023-08-25T04:32:42.918 INFO:teuthology.orchestra.run.smithi162.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 218 kB/s | 44 kB 00:00 2023-08-25T04:32:42.918 INFO:teuthology.orchestra.run.smithi145.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 1.1 MB/s | 47 kB 00:00 2023-08-25T04:32:42.960 INFO:teuthology.orchestra.run.smithi145.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 351 kB/s | 15 kB 00:00 2023-08-25T04:32:43.035 INFO:teuthology.orchestra.run.smithi162.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-08-25T04:32:43.043 INFO:teuthology.orchestra.run.smithi145.stdout:(34/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 3.1 MB/s | 261 kB 00:00 2023-08-25T04:32:43.094 INFO:teuthology.orchestra.run.smithi145.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 3.4 MB/s | 171 kB 00:00 2023-08-25T04:32:43.102 INFO:teuthology.orchestra.run.smithi162.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-08-25T04:32:43.127 INFO:teuthology.orchestra.run.smithi162.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 158 kB/s | 33 kB 00:00 2023-08-25T04:32:43.127 INFO:teuthology.orchestra.run.smithi145.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 541 kB/s | 18 kB 00:00 2023-08-25T04:32:43.152 INFO:teuthology.orchestra.run.smithi162.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2023-08-25T04:32:43.286 INFO:teuthology.orchestra.run.smithi162.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 940 kB/s | 173 kB 00:00 2023-08-25T04:32:43.319 INFO:teuthology.orchestra.run.smithi162.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 487 kB/s | 81 kB 00:00 2023-08-25T04:32:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-08-25T04:32:43.404 INFO:teuthology.orchestra.run.smithi145.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-08-25T04:32:43.428 INFO:teuthology.orchestra.run.smithi162.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-08-25T04:32:43.494 INFO:teuthology.orchestra.run.smithi162.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.0 MB/s | 384 kB 00:00 2023-08-25T04:32:43.512 INFO:teuthology.orchestra.run.smithi145.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 552 kB/s | 59 kB 00:00 2023-08-25T04:32:43.554 INFO:teuthology.orchestra.run.smithi162.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-08-25T04:32:43.570 INFO:teuthology.orchestra.run.smithi145.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-08-25T04:32:43.604 INFO:teuthology.orchestra.run.smithi162.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 535 kB/s | 59 kB 00:00 2023-08-25T04:32:43.712 INFO:teuthology.orchestra.run.smithi162.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 380 kB/s | 60 kB 00:00 2023-08-25T04:32:43.755 INFO:teuthology.orchestra.run.smithi162.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-08-25T04:32:43.780 INFO:teuthology.orchestra.run.smithi162.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-08-25T04:32:43.780 INFO:teuthology.orchestra.run.smithi145.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 6.6 MB/s | 7.4 MB 00:01 2023-08-25T04:32:43.821 INFO:teuthology.orchestra.run.smithi145.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-08-25T04:32:43.822 INFO:teuthology.orchestra.run.smithi162.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-08-25T04:32:43.898 INFO:teuthology.orchestra.run.smithi162.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2023-08-25T04:32:43.922 INFO:teuthology.orchestra.run.smithi162.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-08-25T04:32:43.948 INFO:teuthology.orchestra.run.smithi162.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2023-08-25T04:32:43.954 INFO:teuthology.orchestra.run.smithi145.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 252 kB/s | 44 kB 00:00 2023-08-25T04:32:44.006 INFO:teuthology.orchestra.run.smithi162.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-08-25T04:32:44.015 INFO:teuthology.orchestra.run.smithi145.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 965 kB/s | 185 kB 00:00 2023-08-25T04:32:44.023 INFO:teuthology.orchestra.run.smithi162.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-08-25T04:32:44.099 INFO:teuthology.orchestra.run.smithi162.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 194 kB/s | 29 kB 00:00 2023-08-25T04:32:44.116 INFO:teuthology.orchestra.run.smithi162.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2023-08-25T04:32:44.130 INFO:teuthology.orchestra.run.smithi145.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 188 kB/s | 33 kB 00:00 2023-08-25T04:32:44.166 INFO:teuthology.orchestra.run.smithi162.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.7 MB/s | 255 kB 00:00 2023-08-25T04:32:44.231 INFO:teuthology.orchestra.run.smithi145.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 75 kB/s | 16 kB 00:00 2023-08-25T04:32:44.268 INFO:teuthology.orchestra.run.smithi162.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 403 kB/s | 61 kB 00:00 2023-08-25T04:32:44.291 INFO:teuthology.orchestra.run.smithi162.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 683 kB/s | 85 kB 00:00 2023-08-25T04:32:44.316 INFO:teuthology.orchestra.run.smithi162.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.1 MB/s | 251 kB 00:00 2023-08-25T04:32:44.342 INFO:teuthology.orchestra.run.smithi162.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-25T04:32:44.375 INFO:teuthology.orchestra.run.smithi162.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 214 kB/s | 23 kB 00:00 2023-08-25T04:32:44.481 INFO:teuthology.orchestra.run.smithi145.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-08-25T04:32:44.484 INFO:teuthology.orchestra.run.smithi162.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.0 MB/s | 110 kB 00:00 2023-08-25T04:32:44.526 INFO:teuthology.orchestra.run.smithi162.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-25T04:32:44.573 INFO:teuthology.orchestra.run.smithi145.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 390 kB/s | 173 kB 00:00 2023-08-25T04:32:44.576 INFO:teuthology.orchestra.run.smithi162.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-25T04:32:44.668 INFO:teuthology.orchestra.run.smithi162.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 4.6 MB/s | 1.7 MB 00:00 2023-08-25T04:32:44.702 INFO:teuthology.orchestra.run.smithi162.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2023-08-25T04:32:44.749 INFO:teuthology.orchestra.run.smithi145.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 304 kB/s | 81 kB 00:00 2023-08-25T04:32:44.752 INFO:teuthology.orchestra.run.smithi162.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 465 kB/s | 39 kB 00:00 2023-08-25T04:32:44.774 INFO:teuthology.orchestra.run.smithi145.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 55 kB/s | 11 kB 00:00 2023-08-25T04:32:44.794 INFO:teuthology.orchestra.run.smithi162.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 7.8 MB/s | 688 kB 00:00 2023-08-25T04:32:44.861 INFO:teuthology.orchestra.run.smithi162.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-08-25T04:32:44.886 INFO:teuthology.orchestra.run.smithi162.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-08-25T04:32:44.903 INFO:teuthology.orchestra.run.smithi162.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-08-25T04:32:44.945 INFO:teuthology.orchestra.run.smithi162.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-08-25T04:32:44.987 INFO:teuthology.orchestra.run.smithi162.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 640 kB/s | 27 kB 00:00 2023-08-25T04:32:45.041 INFO:teuthology.orchestra.run.smithi145.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 60 kB/s | 18 kB 00:00 2023-08-25T04:32:45.062 INFO:teuthology.orchestra.run.smithi162.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-08-25T04:32:45.104 INFO:teuthology.orchestra.run.smithi162.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-08-25T04:32:45.174 INFO:teuthology.orchestra.run.smithi145.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 2.9 MB/s | 1.2 MB 00:00 2023-08-25T04:32:45.200 INFO:teuthology.orchestra.run.smithi145.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 308 kB/s | 49 kB 00:00 2023-08-25T04:32:45.246 INFO:teuthology.orchestra.run.smithi162.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-08-25T04:32:45.350 INFO:teuthology.orchestra.run.smithi145.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 335 kB/s | 59 kB 00:00 2023-08-25T04:32:45.375 INFO:teuthology.orchestra.run.smithi145.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 344 kB/s | 60 kB 00:00 2023-08-25T04:32:45.446 INFO:teuthology.orchestra.run.smithi162.stdout:(86/131): python3-scipy-1.0.0-21.module+el8.5.0 12 MB/s | 14 MB 00:01 2023-08-25T04:32:45.475 INFO:teuthology.orchestra.run.smithi145.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-08-25T04:32:45.480 INFO:teuthology.orchestra.run.smithi162.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.9 MB/s | 457 kB 00:00 2023-08-25T04:32:45.500 INFO:teuthology.orchestra.run.smithi145.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 125 kB/s | 16 kB 00:00 2023-08-25T04:32:45.513 INFO:teuthology.orchestra.run.smithi162.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.3 MB/s | 225 kB 00:00 2023-08-25T04:32:45.539 INFO:teuthology.orchestra.run.smithi162.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-08-25T04:32:45.564 INFO:teuthology.orchestra.run.smithi162.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-25T04:32:45.601 INFO:teuthology.orchestra.run.smithi145.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-08-25T04:32:45.639 INFO:teuthology.orchestra.run.smithi162.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-08-25T04:32:45.664 INFO:teuthology.orchestra.run.smithi162.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-08-25T04:32:45.706 INFO:teuthology.orchestra.run.smithi145.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 955 kB/s | 196 kB 00:00 2023-08-25T04:32:45.715 INFO:teuthology.orchestra.run.smithi162.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-08-25T04:32:45.823 INFO:teuthology.orchestra.run.smithi162.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 5.2 MB/s | 3.7 MB 00:00 2023-08-25T04:32:45.843 INFO:teuthology.orchestra.run.smithi145.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 266 kB/s | 64 kB 00:00 2023-08-25T04:32:45.849 INFO:teuthology.orchestra.run.smithi162.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-08-25T04:32:45.891 INFO:teuthology.orchestra.run.smithi162.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-08-25T04:32:45.902 INFO:teuthology.orchestra.run.smithi145.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 96 kB/s | 19 kB 00:00 2023-08-25T04:32:45.941 INFO:teuthology.orchestra.run.smithi162.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-08-25T04:32:46.002 INFO:teuthology.orchestra.run.smithi145.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 151 kB/s | 24 kB 00:00 2023-08-25T04:32:46.044 INFO:teuthology.orchestra.run.smithi145.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 206 kB/s | 29 kB 00:00 2023-08-25T04:32:46.050 INFO:teuthology.orchestra.run.smithi162.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 13 MB/s | 4.9 MB 00:00 2023-08-25T04:32:46.102 INFO:teuthology.orchestra.run.smithi145.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 277 kB/s | 27 kB 00:00 2023-08-25T04:32:46.108 INFO:teuthology.orchestra.run.smithi162.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-08-25T04:32:46.178 INFO:teuthology.orchestra.run.smithi162.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.2 MB/s | 808 kB 00:00 2023-08-25T04:32:46.179 INFO:teuthology.orchestra.run.smithi145.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-08-25T04:32:46.225 INFO:teuthology.orchestra.run.smithi162.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2023-08-25T04:32:46.267 INFO:teuthology.orchestra.run.smithi162.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-08-25T04:32:46.293 INFO:teuthology.orchestra.run.smithi162.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-08-25T04:32:46.295 INFO:teuthology.orchestra.run.smithi145.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-08-25T04:32:46.318 INFO:teuthology.orchestra.run.smithi162.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-08-25T04:32:46.320 INFO:teuthology.orchestra.run.smithi145.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2023-08-25T04:32:46.370 INFO:teuthology.orchestra.run.smithi145.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-08-25T04:32:46.386 INFO:teuthology.orchestra.run.smithi162.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-08-25T04:32:46.404 INFO:teuthology.orchestra.run.smithi145.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-08-25T04:32:46.418 INFO:teuthology.orchestra.run.smithi162.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 480 kB/s | 48 kB 00:00 2023-08-25T04:32:46.429 INFO:teuthology.orchestra.run.smithi145.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-25T04:32:46.444 INFO:teuthology.orchestra.run.smithi162.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-08-25T04:32:46.469 INFO:teuthology.orchestra.run.smithi162.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-08-25T04:32:46.494 INFO:teuthology.orchestra.run.smithi162.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-08-25T04:32:46.496 INFO:teuthology.orchestra.run.smithi145.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 704 kB/s | 47 kB 00:00 2023-08-25T04:32:46.519 INFO:teuthology.orchestra.run.smithi162.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-08-25T04:32:46.538 INFO:teuthology.orchestra.run.smithi145.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-08-25T04:32:46.580 INFO:teuthology.orchestra.run.smithi145.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-08-25T04:32:46.622 INFO:teuthology.orchestra.run.smithi145.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-08-25T04:32:46.636 INFO:teuthology.orchestra.run.smithi162.stdout:(111/131): postgresql-server-10.21-2.module+el8 7.3 MB/s | 5.1 MB 00:00 2023-08-25T04:32:46.662 INFO:teuthology.orchestra.run.smithi162.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-08-25T04:32:46.664 INFO:teuthology.orchestra.run.smithi145.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-08-25T04:32:46.687 INFO:teuthology.orchestra.run.smithi162.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 112 kB/s | 19 kB 00:00 2023-08-25T04:32:46.706 INFO:teuthology.orchestra.run.smithi145.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.1 MB/s | 87 kB 00:00 2023-08-25T04:32:46.712 INFO:teuthology.orchestra.run.smithi162.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-08-25T04:32:46.748 INFO:teuthology.orchestra.run.smithi145.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-08-25T04:32:46.777 INFO:teuthology.orchestra.run.smithi162.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 199 kB/s | 18 kB 00:00 2023-08-25T04:32:46.790 INFO:teuthology.orchestra.run.smithi145.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 4.1 MB/s | 1.7 MB 00:00 2023-08-25T04:32:46.815 INFO:teuthology.orchestra.run.smithi145.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-08-25T04:32:46.874 INFO:teuthology.orchestra.run.smithi145.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-08-25T04:32:46.908 INFO:teuthology.orchestra.run.smithi145.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.4 MB/s | 560 kB 00:00 2023-08-25T04:32:46.933 INFO:teuthology.orchestra.run.smithi145.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-08-25T04:32:46.958 INFO:teuthology.orchestra.run.smithi145.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-08-25T04:32:46.992 INFO:teuthology.orchestra.run.smithi145.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-08-25T04:32:47.017 INFO:teuthology.orchestra.run.smithi145.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-08-25T04:32:47.050 INFO:teuthology.orchestra.run.smithi145.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-08-25T04:32:47.084 INFO:teuthology.orchestra.run.smithi145.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-08-25T04:32:47.126 INFO:teuthology.orchestra.run.smithi145.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-08-25T04:32:47.127 INFO:teuthology.orchestra.run.smithi162.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 1.2 MB/s | 559 kB 00:00 2023-08-25T04:32:47.168 INFO:teuthology.orchestra.run.smithi145.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-08-25T04:32:47.171 INFO:teuthology.orchestra.run.smithi162.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.3 MB/s | 605 kB 00:00 2023-08-25T04:32:47.197 INFO:teuthology.orchestra.run.smithi162.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 567 kB/s | 238 kB 00:00 2023-08-25T04:32:47.222 INFO:teuthology.orchestra.run.smithi162.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-08-25T04:32:47.260 INFO:teuthology.orchestra.run.smithi145.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 11 MB/s | 2.3 MB 00:00 2023-08-25T04:32:47.285 INFO:teuthology.orchestra.run.smithi145.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 512 kB/s | 60 kB 00:00 2023-08-25T04:32:47.289 INFO:teuthology.orchestra.run.smithi162.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 902 kB/s | 83 kB 00:00 2023-08-25T04:32:47.310 INFO:teuthology.orchestra.run.smithi145.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-08-25T04:32:47.314 INFO:teuthology.orchestra.run.smithi162.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 653 kB/s | 60 kB 00:00 2023-08-25T04:32:47.331 INFO:teuthology.orchestra.run.smithi162.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-08-25T04:32:47.335 INFO:teuthology.orchestra.run.smithi145.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-08-25T04:32:47.356 INFO:teuthology.orchestra.run.smithi162.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-08-25T04:32:47.361 INFO:teuthology.orchestra.run.smithi145.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-08-25T04:32:47.382 INFO:teuthology.orchestra.run.smithi162.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-08-25T04:32:47.394 INFO:teuthology.orchestra.run.smithi145.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-08-25T04:32:47.420 INFO:teuthology.orchestra.run.smithi145.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-08-25T04:32:47.435 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2023-08-25T04:32:47.445 INFO:teuthology.orchestra.run.smithi145.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-08-25T04:32:47.462 INFO:teuthology.orchestra.run.smithi162.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.0 MB/s | 644 kB 00:00 2023-08-25T04:32:47.488 INFO:teuthology.orchestra.run.smithi145.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-08-25T04:32:47.499 INFO:teuthology.orchestra.run.smithi162.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.4 MB/s | 171 kB 00:00 2023-08-25T04:32:47.529 INFO:teuthology.orchestra.run.smithi145.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-25T04:32:47.533 INFO:teuthology.orchestra.run.smithi162.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.9 MB/s | 138 kB 00:00 2023-08-25T04:32:47.579 INFO:teuthology.orchestra.run.smithi145.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-25T04:32:47.583 INFO:teuthology.orchestra.run.smithi162.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.1 MB/s | 2.3 MB 00:00 2023-08-25T04:32:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T04:32:47.658 INFO:teuthology.orchestra.run.smithi145.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.1 MB/s | 323 kB 00:00 2023-08-25T04:32:47.675 INFO:teuthology.orchestra.run.smithi162.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 8.5 MB/s | 798 kB 00:00 2023-08-25T04:32:47.696 INFO:teuthology.orchestra.run.smithi145.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.0 MB/s | 39 kB 00:00 2023-08-25T04:32:47.755 INFO:teuthology.orchestra.run.smithi145.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-08-25T04:32:47.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T04:32:47.806 INFO:teuthology.orchestra.run.smithi145.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 10 MB/s | 538 kB 00:00 2023-08-25T04:32:47.848 INFO:teuthology.orchestra.run.smithi145.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-08-25T04:32:47.889 INFO:teuthology.orchestra.run.smithi145.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-08-25T04:32:48.013 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T04:32:48.073 INFO:teuthology.orchestra.run.smithi145.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 21 MB/s | 14 MB 00:00 2023-08-25T04:32:48.115 INFO:teuthology.orchestra.run.smithi145.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.1 MB/s | 713 kB 00:00 2023-08-25T04:32:48.142 INFO:teuthology.orchestra.run.smithi162.stdout:(130/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 6.0 MB/s | 3.6 MB 00:00 2023-08-25T04:32:48.157 INFO:teuthology.orchestra.run.smithi145.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-08-25T04:32:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T04:32:48.207 INFO:teuthology.orchestra.run.smithi145.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-08-25T04:32:48.250 INFO:teuthology.orchestra.run.smithi145.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-08-25T04:32:48.299 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T04:32:48.325 INFO:teuthology.orchestra.run.smithi145.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-08-25T04:32:48.375 INFO:teuthology.orchestra.run.smithi145.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-08-25T04:32:48.467 INFO:teuthology.orchestra.run.smithi145.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-08-25T04:32:48.542 INFO:teuthology.orchestra.run.smithi145.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-08-25T04:32:48.548 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T04:32:48.568 INFO:teuthology.orchestra.run.smithi145.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2023-08-25T04:32:48.593 INFO:teuthology.orchestra.run.smithi145.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-08-25T04:32:48.618 INFO:teuthology.orchestra.run.smithi145.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-08-25T04:32:48.644 INFO:teuthology.orchestra.run.smithi145.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-08-25T04:32:48.962 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T04:32:49.102 INFO:teuthology.orchestra.run.smithi145.stdout:(119/131): ceph-test-18.0.0-5703.gba6b39e4.el8. 5.8 MB/s | 48 MB 00:08 2023-08-25T04:32:49.152 INFO:teuthology.orchestra.run.smithi145.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 1.3 MB/s | 709 kB 00:00 2023-08-25T04:32:49.156 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:32:49.212 INFO:teuthology.orchestra.run.smithi145.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2023-08-25T04:32:49.261 INFO:teuthology.orchestra.run.smithi145.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-08-25T04:32:49.320 INFO:teuthology.orchestra.run.smithi145.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-08-25T04:32:49.333 INFO:teuthology.orchestra.run.smithi162.stdout:(131/131): librados2-18.0.0-5703.gba6b39e4.el8. 2.0 MB/s | 3.7 MB 00:01 2023-08-25T04:32:49.336 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:32:49.336 INFO:teuthology.orchestra.run.smithi162.stdout:Total 10 MB/s | 206 MB 00:20 2023-08-25T04:32:49.454 INFO:teuthology.orchestra.run.smithi145.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 6.0 MB/s | 4.9 MB 00:00 2023-08-25T04:32:49.537 INFO:teuthology.orchestra.run.smithi145.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-08-25T04:32:49.560 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:32:49.596 INFO:teuthology.orchestra.run.smithi145.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-08-25T04:32:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-08-25T04:32:49.756 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:32:49.883 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:32:49.886 INFO:teuthology.orchestra.run.smithi145.stdout:(128/131): postgresql-server-10.21-2.module+el8 9.0 MB/s | 5.1 MB 00:00 2023-08-25T04:32:50.056 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:32:50.137 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:32:50.147 INFO:teuthology.orchestra.run.smithi145.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.0 MB/s | 798 kB 00:00 2023-08-25T04:32:50.181 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-08-25T04:32:50.302 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-08-25T04:32:50.302 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-08-25T04:32:50.340 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T04:32:50.560 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T04:32:50.614 INFO:teuthology.orchestra.run.smithi145.stdout:(130/131): librados2-18.0.0-5703.gba6b39e4.el8. 3.6 MB/s | 3.7 MB 00:01 2023-08-25T04:32:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T04:32:51.022 INFO:teuthology.orchestra.run.smithi145.stdout:(131/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-08-25T04:32:51.025 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:32:51.026 INFO:teuthology.orchestra.run.smithi145.stdout:Total 11 MB/s | 206 MB 00:18 2023-08-25T04:32:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T04:32:51.410 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T04:32:51.438 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-08-25T04:32:51.442 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-08-25T04:32:51.564 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:32:51.667 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:32:51.814 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:32:51.902 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:32:52.056 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T04:32:52.239 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T04:32:52.431 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T04:32:52.676 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-08-25T04:32:52.812 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-08-25T04:32:52.812 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-08-25T04:32:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T04:32:53.239 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T04:32:53.838 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T04:32:54.030 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T04:32:54.350 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-08-25T04:32:54.356 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-08-25T04:32:54.719 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T04:32:54.970 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T04:32:55.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T04:32:55.434 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T04:32:55.653 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T04:32:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T04:32:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T04:32:56.261 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T04:32:56.416 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T04:32:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-08-25T04:32:57.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T04:32:57.550 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T04:32:57.661 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T04:32:57.941 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T04:32:57.943 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T04:32:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T04:32:58.205 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T04:32:58.379 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T04:32:58.396 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T04:32:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T04:32:58.829 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T04:32:59.258 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T04:32:59.505 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-08-25T04:32:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:32:59.725 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T04:32:59.911 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:32:59.936 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T04:33:00.127 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-25T04:33:00.129 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:33:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T04:33:00.258 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:33:00.271 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-25T04:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T04:33:00.462 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:33:00.562 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:33:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T04:33:00.780 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T04:33:00.802 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-25T04:33:00.839 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T04:33:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T04:33:01.100 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T04:33:01.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T04:33:01.264 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-25T04:33:01.361 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T04:33:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T04:33:01.485 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 5/134 2023-08-25T04:33:01.530 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T04:33:01.715 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T04:33:01.794 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T04:33:01.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T04:33:01.964 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 5/134 2023-08-25T04:33:02.080 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T04:33:02.127 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T04:33:02.193 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:33:02.245 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T04:33:02.350 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:33:02.382 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:33:02.512 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:33:02.568 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:33:02.646 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:33:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:33:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T04:33:02.882 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:33:02.959 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T04:33:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T04:33:03.115 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-25T04:33:03.224 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T04:33:03.297 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T04:33:03.383 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T04:33:03.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T04:33:03.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T04:33:03.640 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T04:33:03.883 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T04:33:04.111 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T04:33:04.168 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T04:33:04.227 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T04:33:04.392 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T04:33:04.394 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T04:33:04.453 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T04:33:04.551 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:33:04.630 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T04:33:04.676 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T04:33:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T04:33:04.868 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:33:04.992 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T04:33:05.058 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T04:33:05.077 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T04:33:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T04:33:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T04:33:05.315 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-25T04:33:05.526 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-25T04:33:05.528 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T04:33:05.784 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-25T04:33:05.784 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T04:33:05.925 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-25T04:33:05.972 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T04:33:06.017 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T04:33:06.105 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-25T04:33:06.276 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T04:33:06.292 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-25T04:33:06.309 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T04:33:06.460 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T04:33:06.521 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T04:33:06.620 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T04:33:06.773 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T04:33:06.820 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T04:33:06.888 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T04:33:06.994 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-25T04:33:07.000 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T04:33:07.159 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-25T04:33:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T04:33:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T04:33:07.415 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T04:33:07.549 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T04:33:07.578 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T04:33:07.658 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T04:33:07.768 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T04:33:07.857 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-25T04:33:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T04:33:07.939 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T04:33:08.083 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-25T04:33:08.138 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T04:33:08.312 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T04:33:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T04:33:08.753 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T04:33:08.908 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T04:33:08.921 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-25T04:33:08.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T04:33:09.166 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T04:33:09.199 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-25T04:33:09.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T04:33:09.351 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T04:33:09.418 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-25T04:33:09.521 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T04:33:09.697 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T04:33:09.719 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T04:33:09.741 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T04:33:09.882 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T04:33:09.942 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T04:33:09.972 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T04:33:10.067 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T04:33:10.105 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T04:33:10.260 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T04:33:10.300 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T04:33:10.301 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T04:33:10.525 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T04:33:10.626 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-25T04:33:10.756 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T04:33:10.941 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T04:33:11.192 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T04:33:11.426 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T04:33:11.609 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T04:33:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T04:33:12.041 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T04:33:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T04:33:12.603 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-25T04:33:12.672 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T04:33:12.887 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T04:33:12.920 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T04:33:13.115 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T04:33:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T04:33:13.346 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T04:33:13.493 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T04:33:13.544 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T04:33:13.736 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T04:33:13.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-25T04:33:13.856 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T04:33:13.962 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T04:33:14.104 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-25T04:33:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T04:33:14.138 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T04:33:14.379 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T04:33:14.390 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-25T04:33:14.392 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T04:33:14.633 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T04:33:14.671 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-25T04:33:14.895 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T04:33:15.026 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T04:33:15.130 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T04:33:15.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T04:33:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T04:33:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T04:33:15.434 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:15.573 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T04:33:15.594 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T04:33:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T04:33:15.766 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-25T04:33:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T04:33:15.927 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-25T04:33:16.157 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-25T04:33:16.172 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T04:33:16.247 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-25T04:33:16.500 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-25T04:33:16.527 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T04:33:16.704 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-25T04:33:16.775 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T04:33:16.898 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-25T04:33:16.999 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-25T04:33:17.006 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T04:33:17.218 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-25T04:33:17.272 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T04:33:17.395 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-25T04:33:17.529 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T04:33:17.549 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-25T04:33:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T04:33:17.760 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-25T04:33:17.952 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-25T04:33:18.074 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T04:33:18.138 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-25T04:33:18.370 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-25T04:33:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T04:33:18.612 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-25T04:33:18.656 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T04:33:18.931 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T04:33:19.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-25T04:33:19.140 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T04:33:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:19.343 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T04:33:19.421 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-25T04:33:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:19.558 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T04:33:19.613 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T04:33:19.689 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-25T04:33:19.904 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T04:33:19.945 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-25T04:33:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:20.153 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T04:33:20.184 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-25T04:33:20.341 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T04:33:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:20.426 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T04:33:20.494 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:20.542 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-25T04:33:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T04:33:20.774 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-25T04:33:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T04:33:21.150 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T04:33:21.276 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-25T04:33:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T04:33:21.526 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-25T04:33:21.714 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-25T04:33:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T04:33:21.928 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-25T04:33:22.115 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T04:33:22.276 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-25T04:33:22.296 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T04:33:22.502 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T04:33:22.551 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-25T04:33:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T04:33:22.855 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-25T04:33:22.988 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T04:33:23.117 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-25T04:33:23.216 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T04:33:23.282 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T04:33:23.460 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T04:33:23.514 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T04:33:23.588 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T04:33:23.791 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T04:33:23.864 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T04:33:24.031 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T04:33:24.120 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T04:33:24.227 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T04:33:24.363 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T04:33:24.376 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T04:33:24.603 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T04:33:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T04:33:24.821 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T04:33:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T04:33:25.062 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T04:33:25.133 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T04:33:25.242 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T04:33:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T04:33:25.421 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T04:33:25.665 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T04:33:25.938 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T04:33:26.221 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T04:33:26.514 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T04:33:26.562 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T04:33:26.765 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T04:33:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T04:33:26.983 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T04:33:27.198 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T04:33:27.224 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T04:33:27.558 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T04:33:27.881 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T04:33:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T04:33:28.580 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T04:33:28.925 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T04:33:29.002 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T04:33:29.176 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T04:33:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:29.480 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:29.568 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T04:33:29.839 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T04:33:29.889 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T04:33:30.145 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T04:33:31.044 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T04:33:31.462 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:31.566 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T04:33:31.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T04:33:33.761 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:34.064 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:34.674 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:36.686 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:36.962 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:37.164 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:37.668 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T04:33:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:38.143 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T04:33:38.286 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:39.330 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:39.331 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2023-08-25T04:33:39.331 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2023-08-25T04:33:39.331 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /dev 2023-08-25T04:33:39.331 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /run 2023-08-25T04:33:39.331 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2023-08-25T04:33:39.332 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2023-08-25T04:33:39.332 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2023-08-25T04:33:39.332 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2023-08-25T04:33:39.332 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /dev 2023-08-25T04:33:39.332 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:33:39.689 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:33:41.270 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:33:41.468 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:33:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:33:41.864 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:33:42.319 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:33:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:33:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:33:44.146 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:33:44.329 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:33:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T04:33:44.696 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T04:33:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:33:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:33:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:33:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:33:46.529 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:33:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:33:46.946 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T04:33:47.169 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:33:47.205 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:33:47.354 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:33:47.366 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:33:48.672 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T04:33:48.874 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:33:48.915 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:33:49.079 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:33:49.121 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:33:49.279 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:33:49.641 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:33:49.846 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:33:54.394 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:33:54.693 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T04:33:54.886 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T04:33:55.076 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T04:33:55.968 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T04:33:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:56.094 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2023-08-25T04:33:56.094 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2023-08-25T04:33:56.095 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /dev 2023-08-25T04:33:56.095 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /run 2023-08-25T04:33:56.095 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2023-08-25T04:33:56.095 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2023-08-25T04:33:56.096 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2023-08-25T04:33:56.096 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2023-08-25T04:33:56.096 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /dev 2023-08-25T04:33:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:33:56.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T04:33:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T04:33:56.492 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:33:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T04:33:56.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T04:33:57.045 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T04:33:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:33:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:33:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:33:58.668 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T04:33:58.669 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2023-08-25T04:33:58.669 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2023-08-25T04:33:58.669 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2023-08-25T04:33:58.670 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /run 2023-08-25T04:33:58.670 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2023-08-25T04:33:58.670 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2023-08-25T04:33:58.670 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2023-08-25T04:33:58.671 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2023-08-25T04:33:58.671 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2023-08-25T04:33:58.671 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:33:58.808 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:33:59.084 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:33:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:33:59.254 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:33:59.427 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:33:59.446 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:33:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:33:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:33:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T04:33:59.927 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T04:34:00.276 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T04:34:00.562 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T04:34:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:00.826 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T04:34:00.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T04:34:00.998 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:34:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T04:34:01.142 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T04:34:01.267 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T04:34:01.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T04:34:01.340 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T04:34:01.340 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:34:01.449 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:34:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:01.593 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:34:01.754 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T04:34:01.773 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:01.978 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T04:34:01.984 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T04:34:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:34:02.215 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T04:34:02.265 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:34:02.468 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:02.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:34:02.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T04:34:02.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T04:34:02.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T04:34:02.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T04:34:02.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T04:34:02.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T04:34:02.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T04:34:02.479 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T04:34:02.480 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T04:34:02.481 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T04:34:02.482 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T04:34:02.483 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T04:34:02.484 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T04:34:02.485 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T04:34:02.486 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T04:34:02.487 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T04:34:02.488 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T04:34:02.489 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T04:34:02.490 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T04:34:02.491 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:02.492 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T04:34:02.493 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:34:02.706 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:34:03.669 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:34:03.889 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:34:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:34:04.378 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T04:34:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:34:04.693 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:34:04.898 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:34:04.900 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:04.901 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:48 ago on Fri 25 Aug 2023 04:32:16 AM UTC. 2023-08-25T04:34:04.909 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:34:04.954 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:34:05.161 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T04:34:05.295 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T04:34:05.557 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T04:34:05.798 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:34:06.081 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T04:34:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T04:34:06.330 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:34:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:34:06.505 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:34:06.698 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:34:06.740 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:34:06.898 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:34:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:34:07.365 INFO:teuthology.orchestra.run.smithi183.stdout:Installed products updated. 2023-08-25T04:34:07.437 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T04:34:07.456 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:34:07.579 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:34:07.579 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T04:34:07.580 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.581 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.582 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.583 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T04:34:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: jq-1.6-3.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.585 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:07.586 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T04:34:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T04:34:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T04:34:07.588 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T04:34:07.589 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T04:34:07.589 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T04:34:07.589 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T04:34:07.589 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T04:34:07.589 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T04:34:07.590 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T04:34:07.590 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:07.590 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:07.590 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:07.590 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T04:34:07.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T04:34:07.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T04:34:07.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T04:34:07.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T04:34:07.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T04:34:07.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T04:34:07.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:07.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T04:34:07.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T04:34:07.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T04:34:07.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T04:34:07.598 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T04:34:07.598 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T04:34:07.598 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T04:34:07.598 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T04:34:07.599 INFO:teuthology.orchestra.run.smithi183.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T04:34:07.599 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T04:34:07.599 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T04:34:07.599 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T04:34:07.600 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T04:34:07.600 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T04:34:07.600 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T04:34:07.600 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.600 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T04:34:07.601 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T04:34:07.602 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T04:34:07.602 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:34:07.602 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-08-25T04:34:07.664 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:34:07.768 DEBUG:teuthology.parallel:result is None 2023-08-25T04:34:07.794 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T04:34:08.209 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T04:34:08.466 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:34:08.532 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T04:34:08.714 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:34:08.742 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T04:34:10.192 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T04:34:10.439 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:34:10.526 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T04:34:10.689 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:34:10.761 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T04:34:10.923 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:34:11.344 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T04:34:11.547 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:34:12.049 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:34:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T04:34:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T04:34:12.828 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T04:34:13.754 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T04:34:14.128 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T04:34:14.357 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T04:34:14.575 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T04:34:14.783 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T04:34:14.980 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T04:34:16.588 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T04:34:17.378 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:17.835 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T04:34:18.087 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T04:34:18.210 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-25T04:34:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T04:34:18.451 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 117/134 2023-08-25T04:34:18.708 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-25T04:34:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T04:34:18.920 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-25T04:34:19.130 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T04:34:19.183 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-25T04:34:19.311 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:19.417 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-25T04:34:19.545 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T04:34:19.625 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T04:34:19.777 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T04:34:19.779 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T04:34:19.899 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T04:34:20.153 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T04:34:20.166 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T04:34:20.167 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:20.423 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:20.423 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:20.985 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:21.568 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:21.568 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T04:34:21.568 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T04:34:21.568 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:34:21.569 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T04:34:21.570 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T04:34:21.571 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T04:34:21.572 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T04:34:21.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T04:34:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T04:34:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T04:34:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T04:34:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T04:34:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T04:34:21.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T04:34:21.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T04:34:21.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T04:34:21.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T04:34:21.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T04:34:21.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T04:34:21.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T04:34:21.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T04:34:21.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T04:34:21.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T04:34:21.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T04:34:21.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T04:34:21.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T04:34:21.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T04:34:21.580 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T04:34:21.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T04:34:21.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T04:34:21.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T04:34:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T04:34:21.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T04:34:21.586 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T04:34:21.586 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T04:34:21.586 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T04:34:21.586 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T04:34:21.586 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T04:34:21.587 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T04:34:21.588 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T04:34:21.589 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T04:34:21.590 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T04:34:21.590 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T04:34:21.590 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T04:34:21.590 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:21.590 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T04:34:23.170 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:23.479 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:23.516 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T04:34:23.804 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T04:34:24.037 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T04:34:24.389 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:24.390 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:02:07 ago on Fri 25 Aug 2023 04:32:17 AM UTC. 2023-08-25T04:34:24.401 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T04:34:24.800 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T04:34:25.019 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-25T04:34:25.128 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-25T04:34:25.353 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-25T04:34:25.380 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-25T04:34:25.544 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-25T04:34:25.566 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-25T04:34:25.765 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-25T04:34:25.765 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:25.976 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:25.976 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:26.285 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T04:34:26.519 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:26.774 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:26.774 INFO:teuthology.orchestra.run.smithi145.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-25T04:34:26.775 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:34:26.883 INFO:teuthology.orchestra.run.smithi162.stdout:Installed products updated. 2023-08-25T04:34:27.100 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:34:27.100 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2023-08-25T04:34:27.100 INFO:teuthology.orchestra.run.smithi162.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T04:34:27.101 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.103 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.103 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.103 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.104 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.105 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: jq-1.6-3.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T04:34:27.106 INFO:teuthology.orchestra.run.smithi162.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:27.107 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T04:34:27.108 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T04:34:27.109 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:27.110 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T04:34:27.111 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T04:34:27.112 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T04:34:27.113 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T04:34:27.113 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T04:34:27.113 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T04:34:27.113 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T04:34:27.113 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T04:34:27.114 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T04:34:27.115 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T04:34:27.116 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T04:34:27.117 INFO:teuthology.orchestra.run.smithi162.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T04:34:27.118 INFO:teuthology.orchestra.run.smithi162.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T04:34:27.118 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T04:34:27.119 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T04:34:27.119 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T04:34:27.119 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T04:34:27.119 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T04:34:27.119 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T04:34:27.120 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T04:34:27.121 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T04:34:27.121 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T04:34:27.121 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-08-25T04:34:27.122 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-08-25T04:34:27.293 DEBUG:teuthology.parallel:result is None 2023-08-25T04:34:27.379 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T04:34:27.382 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T04:34:27.383 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T04:34:27.384 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T04:34:27.384 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T04:34:27.384 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T04:34:27.384 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T04:34:27.391 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T04:34:27.391 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T04:34:27.391 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T04:34:27.391 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T04:34:27.392 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T04:34:27.393 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T04:34:27.395 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T04:34:27.396 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-25T04:34:27.397 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-25T04:34:27.398 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-25T04:34:27.399 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-25T04:34:27.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-25T04:34:27.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-25T04:34:27.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-25T04:34:27.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-25T04:34:27.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-25T04:34:27.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-25T04:34:27.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-25T04:34:27.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-25T04:34:27.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-25T04:34:27.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-25T04:34:27.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-25T04:34:27.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-25T04:34:27.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-25T04:34:27.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-25T04:34:27.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T04:34:27.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T04:34:27.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T04:34:27.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T04:34:27.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T04:34:30.148 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T04:34:30.149 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:11 ago on Fri 25 Aug 2023 04:32:19 AM UTC. 2023-08-25T04:34:33.715 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-08-25T04:34:33.953 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:34:33.953 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2023-08-25T04:34:33.954 INFO:teuthology.orchestra.run.smithi145.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T04:34:33.954 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.955 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.956 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.957 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.957 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.957 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.958 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T04:34:33.959 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T04:34:33.960 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T04:34:33.960 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T04:34:33.960 INFO:teuthology.orchestra.run.smithi145.stdout: jq-1.6-3.el8.x86_64 2023-08-25T04:34:33.960 INFO:teuthology.orchestra.run.smithi145.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T04:34:33.961 INFO:teuthology.orchestra.run.smithi145.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T04:34:33.961 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T04:34:33.961 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.961 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.961 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.962 INFO:teuthology.orchestra.run.smithi145.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T04:34:33.962 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.963 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T04:34:33.964 INFO:teuthology.orchestra.run.smithi145.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T04:34:33.965 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T04:34:33.965 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T04:34:33.965 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T04:34:33.965 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:33.966 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T04:34:33.967 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T04:34:33.968 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T04:34:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T04:34:33.970 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T04:34:33.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T04:34:33.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T04:34:33.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.974 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T04:34:33.975 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T04:34:33.976 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T04:34:33.976 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:34:33.977 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-08-25T04:34:34.323 DEBUG:teuthology.parallel:result is None 2023-08-25T04:34:34.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:34.547 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T04:34:34.603 INFO:teuthology.orchestra.run.smithi145.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:34.604 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:34.604 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T04:34:34.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:34.832 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T04:34:34.860 INFO:teuthology.orchestra.run.smithi162.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:34.861 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:34.861 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T04:34:34.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:35.009 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T04:34:35.038 INFO:teuthology.orchestra.run.smithi183.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:35.039 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T04:34:35.039 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T04:34:35.039 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-25T04:34:35.040 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:34:35.041 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T04:34:35.140 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:34:35.141 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T04:34:35.187 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:34:35.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T04:34:35.235 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-25T04:34:35.236 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:34:35.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T04:34:35.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T04:34:35.417 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:34:35.418 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T04:34:35.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T04:34:35.525 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:34:35.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T04:34:35.559 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T04:34:35.631 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-25T04:34:35.632 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:34:35.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T04:34:35.700 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T04:34:35.808 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:34:35.809 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T04:34:35.841 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T04:34:35.911 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:34:35.912 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T04:34:35.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T04:34:36.015 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-25T04:34:36.015 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:34:36.015 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T04:34:36.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T04:34:36.194 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:34:36.195 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T04:34:36.227 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T04:34:36.299 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:34:36.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T04:34:36.332 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T04:34:36.402 INFO:teuthology.run_tasks:Running task cephadm... 2023-08-25T04:34:36.539 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'} 2023-08-25T04:34:36.540 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:36.555 INFO:tasks.cephadm:Cluster fsid is b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:34:36.556 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-08-25T04:34:36.556 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.145', 'mon.b': '172.21.15.162', 'mon.c': '172.21.15.183'} 2023-08-25T04:34:36.556 INFO:tasks.cephadm:First mon is mon.a on smithi145 2023-08-25T04:34:36.557 INFO:tasks.cephadm:First mgr is x 2023-08-25T04:34:36.557 INFO:tasks.cephadm:Normalizing hostnames... 2023-08-25T04:34:36.557 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2023-08-25T04:34:36.632 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2023-08-25T04:34:36.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2023-08-25T04:34:36.699 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-08-25T04:34:36.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:36.852 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72276/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-5703-gba6b39e4', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-5703.gba6b39e4'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-08-24 23:23:18.626495', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023aug3-b5', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-08-25T04:34:37.005 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023aug3-b5, sha1 ba6b39e4f4f77fb95c747099b53db0467d435f56 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T04:34:37.005 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm 2023-08-25T04:34:37.006 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:37.396 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 04:34 /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:37.397 DEBUG:teuthology.orchestra.run.smithi162:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:37.806 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 04:34 /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:37.806 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:38.342 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-rw-r--. 1 ubuntu ubuntu 705688 Aug 25 04:34 /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:38.343 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:38.396 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:38.421 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-25T04:34:38.487 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 on all hosts... 2023-08-25T04:34:38.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T04:34:38.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T04:34:38.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 pull 2023-08-25T04:34:40.041 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T04:34:42.657 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T04:34:42.880 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T04:35:18.642 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-08-25T04:35:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T04:35:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T04:35:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2023-08-25T04:35:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T04:35:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2023-08-25T04:35:18.643 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-08-25T04:35:22.289 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-08-25T04:35:22.289 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T04:35:22.290 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T04:35:22.290 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2023-08-25T04:35:22.290 INFO:teuthology.orchestra.run.smithi162.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T04:35:22.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2023-08-25T04:35:22.291 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-08-25T04:35:22.745 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:35:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T04:35:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef", 2023-08-25T04:35:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2023-08-25T04:35:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e263fcaadb67f0bae2627ea3f04151ef1bf468f74f698991bba88a3316474934" 2023-08-25T04:35:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T04:35:22.747 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:35:22.776 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2023-08-25T04:35:22.848 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2023-08-25T04:35:22.884 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2023-08-25T04:35:22.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2023-08-25T04:35:22.997 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2023-08-25T04:35:23.027 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2023-08-25T04:35:23.060 INFO:tasks.cephadm:Writing seed config... 2023-08-25T04:35:23.061 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-08-25T04:35:23.061 INFO:tasks.cephadm: override: [client] debug client = 20 2023-08-25T04:35:23.062 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-08-25T04:35:23.062 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-08-25T04:35:23.062 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-08-25T04:35:23.062 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-08-25T04:35:23.063 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-08-25T04:35:23.063 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-08-25T04:35:23.063 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-08-25T04:35:23.064 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-08-25T04:35:23.064 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-08-25T04:35:23.064 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-08-25T04:35:23.064 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-08-25T04:35:23.065 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-08-25T04:35:23.065 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-08-25T04:35:23.065 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-08-25T04:35:23.065 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-08-25T04:35:23.065 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-08-25T04:35:23.066 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-08-25T04:35:23.066 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-08-25T04:35:23.066 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-08-25T04:35:23.066 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-08-25T04:35:23.067 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-08-25T04:35:23.067 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-08-25T04:35:23.067 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-08-25T04:35:23.067 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-08-25T04:35:23.067 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-08-25T04:35:23.068 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-08-25T04:35:23.068 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-08-25T04:35:23.068 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-08-25T04:35:23.068 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-08-25T04:35:23.069 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-08-25T04:35:23.069 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-08-25T04:35:23.069 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-08-25T04:35:23.069 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-08-25T04:35:23.070 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-08-25T04:35:23.070 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-08-25T04:35:23.070 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-08-25T04:35:23.071 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-08-25T04:35:23.071 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-08-25T04:35:23.071 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-08-25T04:35:23.071 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-08-25T04:35:23.072 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-08-25T04:35:23.072 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-08-25T04:35:23.073 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-08-25T04:35:23.074 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:35:23.075 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-08-25T04:35:23.119 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b24d70d4-4300-11ee-9b3d-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-08-25T04:35:23.119 DEBUG:teuthology.orchestra.run.smithi145:mon.a> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service 2023-08-25T04:35:23.170 DEBUG:teuthology.orchestra.run.smithi145:mgr.x> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x.service 2023-08-25T04:35:23.220 INFO:tasks.cephadm:Bootstrapping... 2023-08-25T04:35:23.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 -v bootstrap --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.145 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-08-25T04:35:23.236 INFO:journalctl@ceph.mon.a.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:35:23.287 INFO:journalctl@ceph.mgr.x.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:35:23.435 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-08-25T04:35:23.435 INFO:teuthology.orchestra.run.smithi145.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56', '-v', 'bootstrap', '--fsid', 'b24d70d4-4300-11ee-9b3d-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.145', '--skip-admin-label'] 2023-08-25T04:35:23.607 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stdout 4.1.1 2023-08-25T04:35:23.608 INFO:teuthology.orchestra.run.smithi145.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-08-25T04:35:23.608 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying podman|docker is present... 2023-08-25T04:35:23.778 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stdout 4.1.1 2023-08-25T04:35:23.778 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying lvm2 is present... 2023-08-25T04:35:23.778 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying time synchronization is in place... 2023-08-25T04:35:23.793 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-25T04:35:23.793 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-25T04:35:23.806 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-25T04:35:23.806 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:35:23.819 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout enabled 2023-08-25T04:35:23.832 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout active 2023-08-25T04:35:23.832 INFO:teuthology.orchestra.run.smithi145.stdout:Unit chronyd.service is enabled and running 2023-08-25T04:35:23.833 INFO:teuthology.orchestra.run.smithi145.stdout:Repeating the final host check... 2023-08-25T04:35:24.001 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stdout 4.1.1 2023-08-25T04:35:24.001 INFO:teuthology.orchestra.run.smithi145.stdout:podman (/bin/podman) version 4.1.1 is present 2023-08-25T04:35:24.001 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl is present 2023-08-25T04:35:24.002 INFO:teuthology.orchestra.run.smithi145.stdout:lvcreate is present 2023-08-25T04:35:24.014 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-25T04:35:24.015 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-25T04:35:24.027 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-25T04:35:24.027 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:35:24.040 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout enabled 2023-08-25T04:35:24.053 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout active 2023-08-25T04:35:24.054 INFO:teuthology.orchestra.run.smithi145.stdout:Unit chronyd.service is enabled and running 2023-08-25T04:35:24.054 INFO:teuthology.orchestra.run.smithi145.stdout:Host looks OK 2023-08-25T04:35:24.054 INFO:teuthology.orchestra.run.smithi145.stdout:Cluster fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:24.055 INFO:teuthology.orchestra.run.smithi145.stdout:Acquiring lock 140445914868200 on /run/cephadm/b24d70d4-4300-11ee-9b3d-001a4aab830c.lock 2023-08-25T04:35:24.055 INFO:teuthology.orchestra.run.smithi145.stdout:Lock 140445914868200 acquired on /run/cephadm/b24d70d4-4300-11ee-9b3d-001a4aab830c.lock 2023-08-25T04:35:24.055 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying IP 172.21.15.145 port 3300 ... 2023-08-25T04:35:24.055 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying IP 172.21.15.145 port 6789 ... 2023-08-25T04:35:24.055 INFO:teuthology.orchestra.run.smithi145.stdout:Base mon IP(s) is [172.21.15.145:3300, 172.21.15.145:6789], mon addrv is [v2:172.21.15.145:3300,v1:172.21.15.145:6789] 2023-08-25T04:35:24.063 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-08-25T04:35:24.063 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.145 metric 100 2023-08-25T04:35:24.069 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-08-25T04:35:24.072 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-08-25T04:35:24.072 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-08-25T04:35:24.075 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-08-25T04:35:24.075 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-08-25T04:35:24.076 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-25T04:35:24.076 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-08-25T04:35:24.076 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout inet6 fe80::71eb:8a0d:636e:f0c3/64 scope link noprefixroute 2023-08-25T04:35:24.076 INFO:teuthology.orchestra.run.smithi145.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-25T04:35:24.077 INFO:teuthology.orchestra.run.smithi145.stdout:Mon IP `172.21.15.145` is in CIDR network `172.21.0.0/20` 2023-08-25T04:35:24.077 INFO:teuthology.orchestra.run.smithi145.stdout:Mon IP `172.21.15.145` is in CIDR network `172.21.0.0/20` 2023-08-25T04:35:24.077 INFO:teuthology.orchestra.run.smithi145.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-08-25T04:35:24.077 INFO:teuthology.orchestra.run.smithi145.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-08-25T04:35:24.079 INFO:teuthology.orchestra.run.smithi145.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T04:35:24.652 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stdout 95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56... 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Getting image source signatures 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Copying blob sha256:57753a2dbe9e592c0393092b3155420b9dbb1f83f68296cfecb3d2f029e1ad16 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Copying blob sha256:6cfdaf403fb5ea185397a3afe33998eb8d2277a1b4936b2369ceaaceb6049ebf 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Copying config sha256:95c9c69e1344793411bb4e462c28e05fea1317e71809df2f5d3d78facaf76eef 2023-08-25T04:35:24.653 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Writing manifest to image destination 2023-08-25T04:35:24.654 INFO:teuthology.orchestra.run.smithi145.stdout:/bin/podman: stderr Storing signatures 2023-08-25T04:35:25.627 INFO:teuthology.orchestra.run.smithi145.stdout:ceph: stdout ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T04:35:25.627 INFO:teuthology.orchestra.run.smithi145.stdout:Ceph version: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev) 2023-08-25T04:35:25.627 INFO:teuthology.orchestra.run.smithi145.stdout:Extracting ceph user uid/gid from container image... 2023-08-25T04:35:26.400 INFO:teuthology.orchestra.run.smithi145.stdout:stat: stdout 167 167 2023-08-25T04:35:26.401 INFO:teuthology.orchestra.run.smithi145.stdout:Creating initial keys... 2023-08-25T04:35:27.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph-authtool: stdout AQCPL+hkfl/EExAA1KCURkBia6xCSgE8VLuauA== 2023-08-25T04:35:28.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph-authtool: stdout AQCQL+hkn75SBxAA+s7Is6yssYTKL5F8kenGEQ== 2023-08-25T04:35:29.332 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph-authtool: stdout AQCQL+hkHGRtOBAALXJSUakHHaO5AeOhuKFucw== 2023-08-25T04:35:29.334 INFO:teuthology.orchestra.run.smithi145.stdout:Creating initial monmap... 2023-08-25T04:35:30.210 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-25T04:35:30.210 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-08-25T04:35:30.210 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:30.211 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-25T04:35:30.211 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool for a [v2:172.21.15.145:3300,v1:172.21.15.145:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-25T04:35:30.212 INFO:teuthology.orchestra.run.smithi145.stdout:setting min_mon_release = pacific 2023-08-25T04:35:30.212 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: set fsid to b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:30.212 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-25T04:35:30.213 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:35:30.213 INFO:teuthology.orchestra.run.smithi145.stdout:Creating mon... 2023-08-25T04:35:31.086 INFO:teuthology.orchestra.run.smithi145.stdout:create mon.a on 2023-08-25T04:35:31.337 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-08-25T04:35:31.539 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-08-25T04:35:33.034 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target → /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target. 2023-08-25T04:35:33.034 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target → /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target. 2023-08-25T04:35:33.376 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a 2023-08-25T04:35:33.377 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service: Unit ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service not loaded. 2023-08-25T04:35:33.573 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target.wants/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@.service. 2023-08-25T04:35:33.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:33 smithi145 systemd[1]: Starting Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:35:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:34 smithi145 podman[97143]: 2023-08-25T04:35:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:34 smithi145 bash[97143]: 0849327ac9628983d47f6684600cfef52219c391d3748ad94c080dffe9a91278 2023-08-25T04:35:34.388 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T04:35:34.388 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout disabled 2023-08-25T04:35:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T04:35:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:35:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:firewalld.service is not enabled 2023-08-25T04:35:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:Not possible to enable service . firewalld.service is not available 2023-08-25T04:35:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mon to start... 2023-08-25T04:35:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mon... 2023-08-25T04:35:34.683 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:34 smithi145 ceph-mon[97183]: mkfs b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:34.683 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:34 smithi145 ceph-mon[97183]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T04:35:34.684 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:34 smithi145 systemd[1]: Started Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:35:35.390 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout cluster: 2023-08-25T04:35:35.390 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout id: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:35.390 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout services: 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.698453s) 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-08-25T04:35:35.391 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout data: 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout pgs: 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:35.392 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.942+0000 7fe260149700 1 Processor -- start 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.943+0000 7fe260149700 1 -- start start 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.943+0000 7fe260149700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.943+0000 7fe260149700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25807a880 con 0x7fe25807be50 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.944+0000 7fe25dee5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.944+0000 7fe25dee5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36014/0 (socket says 172.21.15.145:36014) 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.944+0000 7fe25dee5700 1 -- 172.21.15.145:0/85799514 learned_addr learned my addr 172.21.15.145:0/85799514 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.944+0000 7fe25dee5700 1 -- 172.21.15.145:0/85799514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25807a9c0 con 0x7fe25807be50 2023-08-25T04:35:35.393 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.944+0000 7fe25dee5700 1 --2- 172.21.15.145:0/85799514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2540098a0 tx=0x7fe254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef0a28914c67ee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe25cee3700 1 -- 172.21.15.145:0/85799514 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe254004160 con 0x7fe25807be50 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe25cee3700 1 -- 172.21.15.145:0/85799514 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2540042c0 con 0x7fe25807be50 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe25cee3700 1 -- 172.21.15.145:0/85799514 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2540045a0 con 0x7fe25807be50 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 -- 172.21.15.145:0/85799514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 msgr2=0x7fe25807a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 --2- 172.21.15.145:0/85799514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2540098a0 tx=0x7fe254009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 -- 172.21.15.145:0/85799514 shutdown_connections 2023-08-25T04:35:35.394 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 --2- 172.21.15.145:0/85799514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25807a2b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 -- 172.21.15.145:0/85799514 >> 172.21.15.145:0/85799514 conn(0x7fe258101750 msgr2=0x7fe258103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 -- 172.21.15.145:0/85799514 shutdown_connections 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.945+0000 7fe260149700 1 -- 172.21.15.145:0/85799514 wait complete. 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.946+0000 7fe260149700 1 Processor -- start 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe260149700 1 -- start start 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe260149700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe260149700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25819bdb0 con 0x7fe25807be50 2023-08-25T04:35:35.395 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe25dee5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe25dee5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36018/0 (socket says 172.21.15.145:36018) 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe25dee5700 1 -- 172.21.15.145:0/1330794257 learned_addr learned my addr 172.21.15.145:0/1330794257 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.947+0000 7fe25dee5700 1 -- 172.21.15.145:0/1330794257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254009550 con 0x7fe25807be50 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe25dee5700 1 --2- 172.21.15.145:0/1330794257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe254004000 tx=0x7fe254004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe254011630 con 0x7fe25807be50 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254011790 con 0x7fe25807be50 2023-08-25T04:35:35.396 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25819bfb0 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe254011a30 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25819c3d0 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe254022070 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.948+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe25401c450 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.949+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258195750 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:34.952+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe25401b450 con 0x7fe25807be50 2023-08-25T04:35:35.397 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.024+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe25802cc10 con 0x7fe25807be50 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.024+0000 7fe24effd700 1 -- 172.21.15.145:0/1330794257 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe25401b5f0 con 0x7fe25807be50 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.025+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 msgr2=0x7fe25819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.025+0000 7fe260149700 1 --2- 172.21.15.145:0/1330794257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe254004000 tx=0x7fe254004750 comp rx=0 tx=0).stop 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.025+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 shutdown_connections 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.025+0000 7fe260149700 1 --2- 172.21.15.145:0/1330794257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe25807be50 0x7fe25819b870 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.025+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 >> 172.21.15.145:0/1330794257 conn(0x7fe258101750 msgr2=0x7fe258103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.026+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 shutdown_connections 2023-08-25T04:35:35.398 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.026+0000 7fe260149700 1 -- 172.21.15.145:0/1330794257 wait complete. 2023-08-25T04:35:35.399 INFO:teuthology.orchestra.run.smithi145.stdout:mon is available 2023-08-25T04:35:35.399 INFO:teuthology.orchestra.run.smithi145.stdout:Assimilating anything we can from ceph.conf... 2023-08-25T04:35:35.655 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T04:35:35.655 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: monmap e1: 1 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} 2023-08-25T04:35:35.656 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: fsmap 2023-08-25T04:35:35.656 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: osdmap e1: 0 total, 0 up, 0 in 2023-08-25T04:35:35.656 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: mgrmap e1: no daemons active 2023-08-25T04:35:35.656 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:35 smithi145 ceph-mon[97183]: from='client.? 172.21.15.145:0/1330794257' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-08-25T04:35:36.472 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [global] 2023-08-25T04:35:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout fsid = b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.145:3300,v1:172.21.15.145:6789] 2023-08-25T04:35:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-25T04:35:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-25T04:35:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-25T04:35:36.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-25T04:35:36.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:36.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-25T04:35:36.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [osd] 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-25T04:35:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.906+0000 7fdefaac9700 1 Processor -- start 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.906+0000 7fdefaac9700 1 -- start start 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.907+0000 7fdefaac9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.907+0000 7fdefaac9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef407a840 con 0x7fdef407be10 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.908+0000 7fdef8865700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.908+0000 7fdef8865700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36022/0 (socket says 172.21.15.145:36022) 2023-08-25T04:35:36.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.908+0000 7fdef8865700 1 -- 172.21.15.145:0/1873252199 learned_addr learned my addr 172.21.15.145:0/1873252199 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:36.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.908+0000 7fdef8865700 1 -- 172.21.15.145:0/1873252199 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef407a980 con 0x7fdef407be10 2023-08-25T04:35:36.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.908+0000 7fdef8865700 1 --2- 172.21.15.145:0/1873252199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdee80098a0 tx=0x7fdee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40444393828f18f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:36.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdef37fe700 1 -- 172.21.15.145:0/1873252199 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee8004160 con 0x7fdef407be10 2023-08-25T04:35:36.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdef37fe700 1 -- 172.21.15.145:0/1873252199 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee80042c0 con 0x7fdef407be10 2023-08-25T04:35:36.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdef37fe700 1 -- 172.21.15.145:0/1873252199 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee80045a0 con 0x7fdef407be10 2023-08-25T04:35:36.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 -- 172.21.15.145:0/1873252199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 msgr2=0x7fdef407a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:36.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 --2- 172.21.15.145:0/1873252199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdee80098a0 tx=0x7fdee8009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:36.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 -- 172.21.15.145:0/1873252199 shutdown_connections 2023-08-25T04:35:36.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 --2- 172.21.15.145:0/1873252199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef407a270 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:36.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 -- 172.21.15.145:0/1873252199 >> 172.21.15.145:0/1873252199 conn(0x7fdef4101750 msgr2=0x7fdef4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:36.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 -- 172.21.15.145:0/1873252199 shutdown_connections 2023-08-25T04:35:36.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.909+0000 7fdefaac9700 1 -- 172.21.15.145:0/1873252199 wait complete. 2023-08-25T04:35:36.481 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.910+0000 7fdefaac9700 1 Processor -- start 2023-08-25T04:35:36.481 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdefaac9700 1 -- start start 2023-08-25T04:35:36.481 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdefaac9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:36.481 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdefaac9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef419bdc0 con 0x7fdef407be10 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdef8865700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdef8865700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36038/0 (socket says 172.21.15.145:36038) 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdef8865700 1 -- 172.21.15.145:0/1192025905 learned_addr learned my addr 172.21.15.145:0/1192025905 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.911+0000 7fdef8865700 1 -- 172.21.15.145:0/1192025905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8009550 con 0x7fdef407be10 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdef8865700 1 --2- 172.21.15.145:0/1192025905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdef407b2e0 tx=0x7fdee8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:36.482 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee8011630 con 0x7fdef407be10 2023-08-25T04:35:36.483 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8011790 con 0x7fdef407be10 2023-08-25T04:35:36.483 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef419bfc0 con 0x7fdef407be10 2023-08-25T04:35:36.483 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee8011a30 con 0x7fdef407be10 2023-08-25T04:35:36.483 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.912+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef419c3e0 con 0x7fdef407be10 2023-08-25T04:35:36.483 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.913+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdee8022070 con 0x7fdef407be10 2023-08-25T04:35:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.913+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdee801c450 con 0x7fdef407be10 2023-08-25T04:35:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.913+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4195790 con 0x7fdef407be10 2023-08-25T04:35:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.916+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fdee801b450 con 0x7fdef407be10 2023-08-25T04:35:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.984+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdef407af20 con 0x7fdef407be10 2023-08-25T04:35:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.989+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdee801b5f0 con 0x7fdef407be10 2023-08-25T04:35:36.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.989+0000 7fdef1ffb700 1 -- 172.21.15.145:0/1192025905 <== mon.0 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fdee80375f0 con 0x7fdef407be10 2023-08-25T04:35:36.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 msgr2=0x7fdef419b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:36.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 --2- 172.21.15.145:0/1192025905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdef407b2e0 tx=0x7fdee8004750 comp rx=0 tx=0).stop 2023-08-25T04:35:36.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 shutdown_connections 2023-08-25T04:35:36.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 --2- 172.21.15.145:0/1192025905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdef407be10 0x7fdef419b880 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:36.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 >> 172.21.15.145:0/1192025905 conn(0x7fdef4101750 msgr2=0x7fdef41030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:36.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 shutdown_connections 2023-08-25T04:35:36.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:35.990+0000 7fdefaac9700 1 -- 172.21.15.145:0/1192025905 wait complete. 2023-08-25T04:35:36.486 INFO:teuthology.orchestra.run.smithi145.stdout:Generating new minimal ceph.conf... 2023-08-25T04:35:36.745 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:36 smithi145 ceph-mon[97183]: from='client.? 172.21.15.145:0/1192025905' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-25T04:35:36.745 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:36 smithi145 ceph-mon[97183]: from='client.? 172.21.15.145:0/1192025905' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-08-25T04:35:37.528 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:37 smithi145 ceph-mon[97183]: from='client.? 172.21.15.145:0/1846896060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:35:37.544 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.014+0000 7ff607774700 1 Processor -- start 2023-08-25T04:35:37.547 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.014+0000 7ff607774700 1 -- start start 2023-08-25T04:35:37.547 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.014+0000 7ff607774700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.014+0000 7ff607774700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60007a860 con 0x7ff60007be30 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.014+0000 7ff605510700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff605510700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36052/0 (socket says 172.21.15.145:36052) 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff605510700 1 -- 172.21.15.145:0/1973271846 learned_addr learned my addr 172.21.15.145:0/1973271846 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff605510700 1 -- 172.21.15.145:0/1973271846 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60007a9a0 con 0x7ff60007be30 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff605510700 1 --2- 172.21.15.145:0/1973271846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098a0 tx=0x7ff5fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571695f6bb461e54 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:37.548 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff5f7fff700 1 -- 172.21.15.145:0/1973271846 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc004160 con 0x7ff60007be30 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff5f7fff700 1 -- 172.21.15.145:0/1973271846 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5fc0042c0 con 0x7ff60007be30 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.015+0000 7ff5f7fff700 1 -- 172.21.15.145:0/1973271846 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc011a00 con 0x7ff60007be30 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 -- 172.21.15.145:0/1973271846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 msgr2=0x7ff60007a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 --2- 172.21.15.145:0/1973271846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098a0 tx=0x7ff5fc009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 -- 172.21.15.145:0/1973271846 shutdown_connections 2023-08-25T04:35:37.549 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 --2- 172.21.15.145:0/1973271846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60007a290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 -- 172.21.15.145:0/1973271846 >> 172.21.15.145:0/1973271846 conn(0x7ff600101750 msgr2=0x7ff600103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 -- 172.21.15.145:0/1973271846 shutdown_connections 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.016+0000 7ff607774700 1 -- 172.21.15.145:0/1973271846 wait complete. 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.017+0000 7ff607774700 1 Processor -- start 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.017+0000 7ff607774700 1 -- start start 2023-08-25T04:35:37.550 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff607774700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff607774700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019c5f0 con 0x7ff60007be30 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff605510700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff605510700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36062/0 (socket says 172.21.15.145:36062) 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff605510700 1 -- 172.21.15.145:0/1846896060 learned_addr learned my addr 172.21.15.145:0/1846896060 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff605510700 1 -- 172.21.15.145:0/1846896060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc009550 con 0x7ff60007be30 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.018+0000 7ff605510700 1 --2- 172.21.15.145:0/1846896060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006af0 tx=0x7ff5fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:37.551 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc003a40 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5fc003ba0 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc01a700 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60019c7f0 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60019cc10 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5fc01ac30 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.019+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff5fc019700 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.020+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600061f50 con 0x7ff60007be30 2023-08-25T04:35:37.552 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.023+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff5fc02de40 con 0x7ff60007be30 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.092+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff60010c780 con 0x7ff60007be30 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.092+0000 7ff5f67fc700 1 -- 172.21.15.145:0/1846896060 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7ff5fc024ad0 con 0x7ff60007be30 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 msgr2=0x7ff60019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 --2- 172.21.15.145:0/1846896060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006af0 tx=0x7ff5fc011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 shutdown_connections 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 --2- 172.21.15.145:0/1846896060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff60007be30 0x7ff60019c0b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:37.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 >> 172.21.15.145:0/1846896060 conn(0x7ff600101750 msgr2=0x7ff600102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:37.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 shutdown_connections 2023-08-25T04:35:37.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:37.093+0000 7ff607774700 1 -- 172.21.15.145:0/1846896060 wait complete. 2023-08-25T04:35:37.554 INFO:teuthology.orchestra.run.smithi145.stdout:Restarting the monitor... 2023-08-25T04:35:37.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:37 smithi145 systemd[1]: Stopping Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:35:38.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a[97159]: 2023-08-25T04:35:37.995+0000 7f0e4b104700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T04:35:38.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a[97159]: 2023-08-25T04:35:37.995+0000 7f0e4b104700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-08-25T04:35:38.731 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:38 smithi145 bash[97520]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a 2023-08-25T04:35:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:38 smithi145 systemd[1]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service: Succeeded. 2023-08-25T04:35:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:38 smithi145 systemd[1]: Stopped Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:35:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:38 smithi145 systemd[1]: Starting Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:35:39.400 INFO:teuthology.orchestra.run.smithi145.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-08-25T04:35:39.403 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 podman[97641]: 2023-08-25T04:35:39.403 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 bash[97641]: 60fb508e7675c377d225a194fcef31366b74289edfbd72cab60af9fa833041b6 2023-08-25T04:35:39.403 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 systemd[1]: Started Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:35:39.403 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T04:35:39.482 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T04:35:39.482 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: pidfile_write: ignore empty --pid-file 2023-08-25T04:35:39.482 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: load: jerasure load: lrc 2023-08-25T04:35:39.482 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: RocksDB version: 7.9.2 2023-08-25T04:35:39.482 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Git sha 0 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: DB SUMMARY 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: DB Session ID: E5PU5VWETEJPC5BE82K7 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: CURRENT file: CURRENT 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-08-25T04:35:39.483 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99367 ; 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.error_if_exists: 0 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.create_if_missing: 0 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.paranoid_checks: 1 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T04:35:39.484 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.env: 0x5560297a8da0 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.info_log: 0x55602a3bb440 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.statistics: (nil) 2023-08-25T04:35:39.485 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.use_fsync: 0 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_log_file_size: 0 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_fallocate: 1 2023-08-25T04:35:39.486 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.use_direct_reads: 0 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.db_log_dir: 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.wal_dir: 2023-08-25T04:35:39.487 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T04:35:39.488 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.write_buffer_manager: 0x55602a3754a0 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T04:35:39.489 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.unordered_write: 0 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.row_cache: None 2023-08-25T04:35:39.490 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.wal_filter: None 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.two_write_queues: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.wal_compression: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.atomic_flush: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T04:35:39.491 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.log_readahead_size: 0 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T04:35:39.492 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_background_jobs: 2 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_background_compactions: -1 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_subcompactions: 1 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T04:35:39.493 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T04:35:39.494 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_open_files: -1 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_background_flushes: -1 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Compression algorithms supported: 2023-08-25T04:35:39.495 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kZSTD supported: 0 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kXpressCompression supported: 0 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kZlibCompression supported: 1 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kSnappyCompression supported: 1 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kLZ4Compression supported: 1 2023-08-25T04:35:39.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: kBZip2Compression supported: 0 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.merge_operator: 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_filter: None 2023-08-25T04:35:39.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55602a3bb540) 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout: cache_index_and_filter_blocks: 1 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T04:35:39.498 INFO:journalctl@ceph.mon.a.smithi145.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T04:35:39.499 INFO:journalctl@ceph.mon.a.smithi145.stdout: pin_top_level_index_and_filter: 1 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: index_type: 0 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: data_block_index_type: 0 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: index_shortening: 1 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: checksum: 4 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: no_block_cache: 0 2023-08-25T04:35:39.500 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_cache: 0x55602a4131f0 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_cache_name: BinnedLRUCache 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_cache_options: 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: capacity : 536870912 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: num_shard_bits : 4 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: strict_capacity_limit : 0 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: high_pri_pool_ratio: 0.000 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_cache_compressed: (nil) 2023-08-25T04:35:39.501 INFO:journalctl@ceph.mon.a.smithi145.stdout: persistent_cache: (nil) 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_size: 4096 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_size_deviation: 10 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_restart_interval: 16 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: index_block_restart_interval: 1 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: metadata_block_size: 4096 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: partition_filters: 0 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: use_delta_encoding: 1 2023-08-25T04:35:39.502 INFO:journalctl@ceph.mon.a.smithi145.stdout: filter_policy: bloomfilter 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: whole_key_filtering: 1 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: verify_compression: 0 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: read_amp_bytes_per_bit: 0 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: format_version: 5 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: enable_index_compression: 1 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: block_align: 0 2023-08-25T04:35:39.503 INFO:journalctl@ceph.mon.a.smithi145.stdout: max_auto_readahead_size: 262144 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout: prepopulate_block_cache: 0 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout: initial_auto_readahead_size: 8192 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression: NoCompression 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T04:35:39.504 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.num_levels: 7 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T04:35:39.505 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T04:35:39.506 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T04:35:39.507 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T04:35:39.508 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T04:35:39.509 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T04:35:39.510 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T04:35:39.511 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.table_properties_collectors: 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.inplace_update_support: 0 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T04:35:39.512 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.bloom_locality: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.max_successive_merges: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T04:35:39.513 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.ttl: 2592000 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enable_blob_files: false 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.min_blob_size: 0 2023-08-25T04:35:39.514 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T04:35:39.515 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2d0bcb8a-7fb3-4004-8c0e-4c74cbbfb483 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938139412100, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938139413958, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94567, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92565, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13773, "raw_average_key_size": 50, "raw_value_size": 85055, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692938139, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2d0bcb8a-7fb3-4004-8c0e-4c74cbbfb483", "db_session_id": "E5PU5VWETEJPC5BE82K7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938139414094, "job": 1, "event": "recovery_finished"} 2023-08-25T04:35:39.516 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55602a4b0000 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: DB pointer 0x55602a49c000 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout: ** DB Stats ** 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:35:39.517 INFO:journalctl@ceph.mon.a.smithi145.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.518 INFO:journalctl@ceph.mon.a.smithi145.stdout: ** Compaction Stats [default] ** 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: L0 2/0 94.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: Sum 2/0 94.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: ** Compaction Stats [default] ** 2023-08-25T04:35:39.519 INFO:journalctl@ceph.mon.a.smithi145.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T04:35:39.520 INFO:journalctl@ceph.mon.a.smithi145.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: Cumulative compaction: 0.00 GB write, 14.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: Interval compaction: 0.00 GB write, 14.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T04:35:39.521 INFO:journalctl@ceph.mon.a.smithi145.stdout: Block cache BinnedLRUCache@0x55602a4131f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: starting mon.a rank 0 at public addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] at bind addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???) e1 preinit fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).mds e1 new map 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).mds e1 print_map 2023-08-25T04:35:39.522 INFO:journalctl@ceph.mon.a.smithi145.stdout: e1 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout: legacy client fscid: -1 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout: 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout: No filesystems configured 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T04:35:39.523 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expand map: {default=false} 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta from 'false' to 'false' 2023-08-25T04:35:39.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expanded map: {default=false} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expand map: {default=info} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta from 'info' to 'info' 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expanded map: {default=info} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expand map: {default=daemon} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expand map: {default=debug} 2023-08-25T04:35:39.525 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: expand_channel_meta expanded map: {default=debug} 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: monmap e1: 1 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: fsmap 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: osdmap e1: 0 total, 0 up, 0 in 2023-08-25T04:35:39.526 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:39 smithi145 ceph-mon[97681]: mgrmap e1: no daemons active 2023-08-25T04:35:40.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.045+0000 7f6669467700 1 Processor -- start 2023-08-25T04:35:40.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.045+0000 7f6669467700 1 -- start start 2023-08-25T04:35:40.484 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.045+0000 7f6669467700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.045+0000 7f6669467700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664106ba0 con 0x7f66641061b0 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.046+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.046+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36072/0 (socket says 172.21.15.145:36072) 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.046+0000 7f6662ffd700 1 -- 172.21.15.145:0/1435075597 learned_addr learned my addr 172.21.15.145:0/1435075597 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.047+0000 7f6662ffd700 1 -- 172.21.15.145:0/1435075597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66641073b0 con 0x7f66641061b0 2023-08-25T04:35:40.485 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.047+0000 7f6662ffd700 1 --2- 172.21.15.145:0/1435075597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66540098a0 tx=0x7f6654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a27623416a55849 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:40.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.047+0000 7f6661ffb700 1 -- 172.21.15.145:0/1435075597 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654004160 con 0x7f66641061b0 2023-08-25T04:35:40.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.047+0000 7f6661ffb700 1 -- 172.21.15.145:0/1435075597 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66540042c0 con 0x7f66641061b0 2023-08-25T04:35:40.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.047+0000 7f6661ffb700 1 -- 172.21.15.145:0/1435075597 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654011a00 con 0x7f66641061b0 2023-08-25T04:35:40.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 -- 172.21.15.145:0/1435075597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 msgr2=0x7f66641065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:40.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 --2- 172.21.15.145:0/1435075597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66540098a0 tx=0x7f6654009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 -- 172.21.15.145:0/1435075597 shutdown_connections 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 --2- 172.21.15.145:0/1435075597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f66641065d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 -- 172.21.15.145:0/1435075597 >> 172.21.15.145:0/1435075597 conn(0x7f6664101750 msgr2=0x7f6664103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 -- 172.21.15.145:0/1435075597 shutdown_connections 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.048+0000 7f6669467700 1 -- 172.21.15.145:0/1435075597 wait complete. 2023-08-25T04:35:40.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.049+0000 7f6669467700 1 Processor -- start 2023-08-25T04:35:40.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.049+0000 7f6669467700 1 -- start start 2023-08-25T04:35:40.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.049+0000 7f6669467700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:40.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.049+0000 7f6669467700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419a3e0 con 0x7f66641061b0 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.050+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.050+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36084/0 (socket says 172.21.15.145:36084) 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.050+0000 7f6662ffd700 1 -- 172.21.15.145:0/3635706505 learned_addr learned my addr 172.21.15.145:0/3635706505 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.050+0000 7f6662ffd700 1 -- 172.21.15.145:0/3635706505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009550 con 0x7f66641061b0 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.050+0000 7f6662ffd700 1 --2- 172.21.15.145:0/3635706505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6654006af0 tx=0x7f6654011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.051+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654003a40 con 0x7f66641061b0 2023-08-25T04:35:40.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.051+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6654003ba0 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.051+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666419a5e0 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.051+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f665401a700 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.051+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666419aa00 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.052+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6654024430 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.052+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f665402d730 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.052+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6664061f50 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.055+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6654024870 con 0x7f66641061b0 2023-08-25T04:35:40.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.124+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f666419d440 con 0x7f66641061b0 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.127+0000 7f664bfff700 1 -- 172.21.15.145:0/3635706505 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f665401ade0 con 0x7f66641061b0 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 msgr2=0x7f6664199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 --2- 172.21.15.145:0/3635706505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6654006af0 tx=0x7f6654011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 shutdown_connections 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 --2- 172.21.15.145:0/3635706505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66641061b0 0x7f6664199ea0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 >> 172.21.15.145:0/3635706505 conn(0x7f6664101750 msgr2=0x7f6664102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:40.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 shutdown_connections 2023-08-25T04:35:40.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:40.128+0000 7f6669467700 1 -- 172.21.15.145:0/3635706505 wait complete. 2023-08-25T04:35:40.496 INFO:teuthology.orchestra.run.smithi145.stdout:Wrote config to /etc/ceph/ceph.conf 2023-08-25T04:35:40.520 INFO:teuthology.orchestra.run.smithi145.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-08-25T04:35:40.523 INFO:teuthology.orchestra.run.smithi145.stdout:Creating mgr... 2023-08-25T04:35:40.523 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying port 0.0.0.0:9283 ... 2023-08-25T04:35:40.524 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying port 0.0.0.0:8765 ... 2023-08-25T04:35:40.881 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x 2023-08-25T04:35:40.882 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x.service: Unit ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x.service not loaded. 2023-08-25T04:35:41.079 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c.target.wants/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@.service. 2023-08-25T04:35:41.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:41 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3635706505' entity='client.admin' 2023-08-25T04:35:41.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:41 smithi145 systemd[1]: Starting Ceph mgr.x for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:35:41.830 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:41 smithi145 podman[97913]: 2023-08-25T04:35:41.831 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:41 smithi145 bash[97913]: dd74c19a970f07c8c7c79bfc823d62a52adec42600d739764309fe8a7b4aa770 2023-08-25T04:35:41.881 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T04:35:41.881 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout disabled 2023-08-25T04:35:41.900 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T04:35:41.901 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:35:41.901 INFO:teuthology.orchestra.run.smithi145.stdout:firewalld.service is not enabled 2023-08-25T04:35:41.901 INFO:teuthology.orchestra.run.smithi145.stdout:Not possible to enable service . firewalld.service is not available 2023-08-25T04:35:41.922 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T04:35:41.922 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout disabled 2023-08-25T04:35:41.938 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T04:35:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:35:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:firewalld.service is not enabled 2023-08-25T04:35:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-08-25T04:35:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mgr to start... 2023-08-25T04:35:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mgr... 2023-08-25T04:35:42.203 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:41 smithi145 systemd[1]: Started Ceph mgr.x for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:35:42.204 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:41.973+0000 7fa8f7a5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:35:42.204 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:42.082+0000 7fa8f7a5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:35:42.204 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:42.202+0000 7fa8f7a5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:35:42.567 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:42.331+0000 7fa8f7a5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:35:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:42 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1485394068' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T04:35:42.829 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:42.566+0000 7fa8f7a5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:35:43.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:43.059+0000 7fa8f7a5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:35:43.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:43.221+0000 7fa8f7a5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:35:43.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:43.338+0000 7fa8f7a5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:35:43.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:43.460+0000 7fa8f7a5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:35:44.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:44.002+0000 7fa8f7a5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:35:45.330 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:44.918+0000 7fa8f7a5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:35:45.330 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:45.023+0000 7fa8f7a5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:35:45.435 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:45.435 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:35:45.435 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.436 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 0 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-08-25T04:35:45.437 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:45.438 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T04:35:45.439 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T04:35:45.440 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T04:35:45.441 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T04:35:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T04:35:45.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:45.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T04:35:34.328656+0000", 2023-08-25T04:35:45.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:45.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.486+0000 7f25bf360700 1 Processor -- start 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.486+0000 7f25bf360700 1 -- start start 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.486+0000 7f25bf360700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.486+0000 7f25bf360700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b807a860 con 0x7f25b807be30 2023-08-25T04:35:45.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.487+0000 7f25bd0fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:45.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.487+0000 7f25bd0fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36108/0 (socket says 172.21.15.145:36108) 2023-08-25T04:35:45.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.487+0000 7f25bd0fc700 1 -- 172.21.15.145:0/323352776 learned_addr learned my addr 172.21.15.145:0/323352776 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:45.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.487+0000 7f25bd0fc700 1 -- 172.21.15.145:0/323352776 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b807a9a0 con 0x7f25b807be30 2023-08-25T04:35:45.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.487+0000 7f25bd0fc700 1 --2- 172.21.15.145:0/323352776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25b40098a0 tx=0x7f25b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c488bbf4c7b7cc42 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:45.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25affff700 1 -- 172.21.15.145:0/323352776 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b4004160 con 0x7f25b807be30 2023-08-25T04:35:45.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25affff700 1 -- 172.21.15.145:0/323352776 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25b40042c0 con 0x7f25b807be30 2023-08-25T04:35:45.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25affff700 1 -- 172.21.15.145:0/323352776 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b4011a00 con 0x7f25b807be30 2023-08-25T04:35:45.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 -- 172.21.15.145:0/323352776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 msgr2=0x7f25b807a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:45.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 --2- 172.21.15.145:0/323352776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25b40098a0 tx=0x7f25b4009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:45.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 -- 172.21.15.145:0/323352776 shutdown_connections 2023-08-25T04:35:45.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 --2- 172.21.15.145:0/323352776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b807a290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:45.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 -- 172.21.15.145:0/323352776 >> 172.21.15.145:0/323352776 conn(0x7f25b8101750 msgr2=0x7f25b8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:45.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.488+0000 7f25bf360700 1 -- 172.21.15.145:0/323352776 shutdown_connections 2023-08-25T04:35:45.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.489+0000 7f25bf360700 1 -- 172.21.15.145:0/323352776 wait complete. 2023-08-25T04:35:45.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.489+0000 7f25bf360700 1 Processor -- start 2023-08-25T04:35:45.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bf360700 1 -- start start 2023-08-25T04:35:45.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bf360700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:45.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bf360700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b819c5f0 con 0x7f25b807be30 2023-08-25T04:35:45.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bd0fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:45.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bd0fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36124/0 (socket says 172.21.15.145:36124) 2023-08-25T04:35:45.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.490+0000 7f25bd0fc700 1 -- 172.21.15.145:0/1485394068 learned_addr learned my addr 172.21.15.145:0/1485394068 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:45.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25bd0fc700 1 -- 172.21.15.145:0/1485394068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4009550 con 0x7f25b807be30 2023-08-25T04:35:45.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25bd0fc700 1 --2- 172.21.15.145:0/1485394068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006af0 tx=0x7f25b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:45.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b4003a40 con 0x7f25b807be30 2023-08-25T04:35:45.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25b4003ba0 con 0x7f25b807be30 2023-08-25T04:35:45.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b819c7f0 con 0x7f25b807be30 2023-08-25T04:35:45.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b401a700 con 0x7f25b807be30 2023-08-25T04:35:45.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.491+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b819cc10 con 0x7f25b807be30 2023-08-25T04:35:45.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.493+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f25b401a980 con 0x7f25b807be30 2023-08-25T04:35:45.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.493+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f25b402dd60 con 0x7f25b807be30 2023-08-25T04:35:45.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.493+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8061f50 con 0x7f25b807be30 2023-08-25T04:35:45.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.495+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f25b40248d0 con 0x7f25b807be30 2023-08-25T04:35:45.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.570+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f25b810c770 con 0x7f25b807be30 2023-08-25T04:35:45.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.571+0000 7f25ae7fc700 1 -- 172.21.15.145:0/1485394068 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f25b401ad70 con 0x7f25b807be30 2023-08-25T04:35:45.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 msgr2=0x7f25b819c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:45.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 --2- 172.21.15.145:0/1485394068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006af0 tx=0x7f25b4011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:45.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 shutdown_connections 2023-08-25T04:35:45.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 --2- 172.21.15.145:0/1485394068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25b807be30 0x7f25b819c0b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:45.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 >> 172.21.15.145:0/1485394068 conn(0x7f25b8101750 msgr2=0x7f25b8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:45.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 shutdown_connections 2023-08-25T04:35:45.465 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:42.572+0000 7f25bf360700 1 -- 172.21.15.145:0/1485394068 wait complete. 2023-08-25T04:35:45.465 INFO:teuthology.orchestra.run.smithi145.stdout:mgr not available, waiting (1/15)... 2023-08-25T04:35:46.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:45.985+0000 7fa8f7a5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:35:46.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:46.104+0000 7fa8f7a5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:35:46.685 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:46.327+0000 7fa8f7a5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:35:46.960 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:46.683+0000 7fa8f7a5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:35:47.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:46.958+0000 7fa8f7a5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:35:47.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:47.149+0000 7fa8f7a5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:35:47.793 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:47.517+0000 7fa8f7a5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:35:48.499 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:48 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2571211893' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T04:35:48.539 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:48.539 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:35:48.539 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:35:48.540 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T04:35:48.540 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T04:35:48.540 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T04:35:48.540 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T04:35:48.553 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 0 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T04:35:48.554 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T04:35:48.555 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T04:35:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T04:35:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T04:35:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T04:35:48.557 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.558 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T04:35:48.559 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:48.559 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T04:35:48.559 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T04:35:48.559 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.559 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T04:35:48.560 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-25T04:35:48.560 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T04:35:48.560 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T04:35:48.561 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T04:35:48.561 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T04:35:48.561 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T04:35:48.561 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:48.562 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:48.562 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.562 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T04:35:48.562 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:48.563 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T04:35:34.328656+0000", 2023-08-25T04:35:48.563 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:48.563 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:48.564 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T04:35:48.564 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:35:48.564 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.018+0000 7fe99333b700 1 Processor -- start 2023-08-25T04:35:48.564 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.018+0000 7fe99333b700 1 -- start start 2023-08-25T04:35:48.565 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.019+0000 7fe99333b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:48.565 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.019+0000 7fe99333b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c07a860 con 0x7fe98c07be30 2023-08-25T04:35:48.565 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.019+0000 7fe9910d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:48.566 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.019+0000 7fe9910d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36604/0 (socket says 172.21.15.145:36604) 2023-08-25T04:35:48.566 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.019+0000 7fe9910d7700 1 -- 172.21.15.145:0/3799130853 learned_addr learned my addr 172.21.15.145:0/3799130853 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:48.566 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.020+0000 7fe9910d7700 1 -- 172.21.15.145:0/3799130853 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c07a9a0 con 0x7fe98c07be30 2023-08-25T04:35:48.566 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.020+0000 7fe9910d7700 1 --2- 172.21.15.145:0/3799130853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9880098a0 tx=0x7fe988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7356ab08309446dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:48.567 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.020+0000 7fe983fff700 1 -- 172.21.15.145:0/3799130853 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe988004160 con 0x7fe98c07be30 2023-08-25T04:35:48.567 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.020+0000 7fe983fff700 1 -- 172.21.15.145:0/3799130853 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9880042c0 con 0x7fe98c07be30 2023-08-25T04:35:48.567 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.020+0000 7fe983fff700 1 -- 172.21.15.145:0/3799130853 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe988011a00 con 0x7fe98c07be30 2023-08-25T04:35:48.567 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 -- 172.21.15.145:0/3799130853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 msgr2=0x7fe98c07a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:48.568 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 --2- 172.21.15.145:0/3799130853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9880098a0 tx=0x7fe988009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:48.568 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 -- 172.21.15.145:0/3799130853 shutdown_connections 2023-08-25T04:35:48.568 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 --2- 172.21.15.145:0/3799130853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c07a290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:48.569 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 -- 172.21.15.145:0/3799130853 >> 172.21.15.145:0/3799130853 conn(0x7fe98c101750 msgr2=0x7fe98c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:48.569 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 -- 172.21.15.145:0/3799130853 shutdown_connections 2023-08-25T04:35:48.569 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.021+0000 7fe99333b700 1 -- 172.21.15.145:0/3799130853 wait complete. 2023-08-25T04:35:48.569 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.022+0000 7fe99333b700 1 Processor -- start 2023-08-25T04:35:48.569 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.022+0000 7fe99333b700 1 -- start start 2023-08-25T04:35:48.570 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe99333b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:48.570 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe99333b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c19c5f0 con 0x7fe98c07be30 2023-08-25T04:35:48.570 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe9910d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:48.570 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe9910d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36610/0 (socket says 172.21.15.145:36610) 2023-08-25T04:35:48.570 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe9910d7700 1 -- 172.21.15.145:0/2571211893 learned_addr learned my addr 172.21.15.145:0/2571211893 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:48.571 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.023+0000 7fe9910d7700 1 -- 172.21.15.145:0/2571211893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009550 con 0x7fe98c07be30 2023-08-25T04:35:48.571 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe9910d7700 1 --2- 172.21.15.145:0/2571211893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe988006af0 tx=0x7fe988011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:48.571 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe988003a40 con 0x7fe98c07be30 2023-08-25T04:35:48.571 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe988003ba0 con 0x7fe98c07be30 2023-08-25T04:35:48.571 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c19c7f0 con 0x7fe98c07be30 2023-08-25T04:35:48.572 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98801a700 con 0x7fe98c07be30 2023-08-25T04:35:48.572 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.024+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c19cc10 con 0x7fe98c07be30 2023-08-25T04:35:48.572 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.025+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe98801ac30 con 0x7fe98c07be30 2023-08-25T04:35:48.572 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.025+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe988019700 con 0x7fe98c07be30 2023-08-25T04:35:48.572 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.025+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c061f50 con 0x7fe98c07be30 2023-08-25T04:35:48.573 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.028+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe98802de40 con 0x7fe98c07be30 2023-08-25T04:35:48.573 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.104+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe98c10c780 con 0x7fe98c07be30 2023-08-25T04:35:48.573 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.105+0000 7fe9827fc700 1 -- 172.21.15.145:0/2571211893 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fe988024ad0 con 0x7fe98c07be30 2023-08-25T04:35:48.573 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 msgr2=0x7fe98c19c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:48.573 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 --2- 172.21.15.145:0/2571211893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe988006af0 tx=0x7fe988011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 shutdown_connections 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 --2- 172.21.15.145:0/2571211893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe98c07be30 0x7fe98c19c0b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 >> 172.21.15.145:0/2571211893 conn(0x7fe98c101750 msgr2=0x7fe98c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 shutdown_connections 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:48.106+0000 7fe99333b700 1 -- 172.21.15.145:0/2571211893 wait complete. 2023-08-25T04:35:48.574 INFO:teuthology.orchestra.run.smithi145.stdout:mgr not available, waiting (2/15)... 2023-08-25T04:35:48.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:48.497+0000 7fa8f7a5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:35:48.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:48.673+0000 7fa8f7a5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:35:49.329 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:48.917+0000 7fa8f7a5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:35:49.329 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:49.059+0000 7fa8f7a5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:35:49.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:49.330+0000 7fa8f7a5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:35:50.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: Activating manager daemon x 2023-08-25T04:35:50.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: mgrmap e2: x(active, starting, since 0.00447641s) 2023-08-25T04:35:50.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:35:50.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:35:50.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:35:50.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:35:50.829 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:50.413+0000 7fa8f7a5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:35:51.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:35:51.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: Manager daemon x is now available 2023-08-25T04:35:51.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:35:51.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' 2023-08-25T04:35:51.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:35:51.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' 2023-08-25T04:35:51.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:51 smithi145 ceph-mon[97681]: from='mgr.14100 172.21.15.145:0/2208388468' entity='mgr.x' 2023-08-25T04:35:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:52 smithi145 ceph-mon[97681]: mgrmap e3: x(active, since 1.01983s) 2023-08-25T04:35:53.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:53 smithi145 ceph-mon[97681]: mgrmap e4: x(active, since 2s) 2023-08-25T04:35:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3644460860' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-25T04:35:56.434 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:56.443 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:35:56.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:35:56.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "health": { 2023-08-25T04:35:56.444 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-25T04:35:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-25T04:35:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-25T04:35:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-25T04:35:56.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-25T04:35:56.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 0 2023-08-25T04:35:56.446 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:56.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-25T04:35:56.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "a" 2023-08-25T04:35:56.447 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:56.448 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-08-25T04:35:56.448 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-25T04:35:56.448 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:56.448 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-25T04:35:56.449 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.450 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-25T04:35:56.451 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-25T04:35:56.452 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-25T04:35:56.453 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "restful" 2023-08-25T04:35:56.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ], 2023-08-25T04:35:56.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:56.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.454 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-25T04:35:56.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-25T04:35:56.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "modified": "2023-08-25T04:35:34.328656+0000", 2023-08-25T04:35:56.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-25T04:35:56.455 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout }, 2023-08-25T04:35:56.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-25T04:35:56.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:35:56.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.670+0000 7fd968909700 1 Processor -- start 2023-08-25T04:35:56.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd968909700 1 -- start start 2023-08-25T04:35:56.456 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd968909700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:56.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd968909700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd964106ba0 con 0x7fd9641061b0 2023-08-25T04:35:56.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd96259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:56.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd96259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54932/0 (socket says 172.21.15.145:54932) 2023-08-25T04:35:56.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.671+0000 7fd96259c700 1 -- 172.21.15.145:0/492228173 learned_addr learned my addr 172.21.15.145:0/492228173 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:56.457 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.672+0000 7fd96259c700 1 -- 172.21.15.145:0/492228173 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9641073b0 con 0x7fd9641061b0 2023-08-25T04:35:56.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.672+0000 7fd96259c700 1 --2- 172.21.15.145:0/492228173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098a0 tx=0x7fd954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264a277f03a61249 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:56.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.672+0000 7fd96159a700 1 -- 172.21.15.145:0/492228173 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd954004160 con 0x7fd9641061b0 2023-08-25T04:35:56.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.672+0000 7fd96159a700 1 -- 172.21.15.145:0/492228173 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9540042c0 con 0x7fd9641061b0 2023-08-25T04:35:56.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.672+0000 7fd96159a700 1 -- 172.21.15.145:0/492228173 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd954011a00 con 0x7fd9641061b0 2023-08-25T04:35:56.458 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 -- 172.21.15.145:0/492228173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 msgr2=0x7fd9641065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 --2- 172.21.15.145:0/492228173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098a0 tx=0x7fd954009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 -- 172.21.15.145:0/492228173 shutdown_connections 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 --2- 172.21.15.145:0/492228173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd9641065d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 -- 172.21.15.145:0/492228173 >> 172.21.15.145:0/492228173 conn(0x7fd964101750 msgr2=0x7fd964103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 -- 172.21.15.145:0/492228173 shutdown_connections 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.673+0000 7fd968909700 1 -- 172.21.15.145:0/492228173 wait complete. 2023-08-25T04:35:56.459 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.674+0000 7fd968909700 1 Processor -- start 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd968909700 1 -- start start 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd968909700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd968909700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96407c120 con 0x7fd9641061b0 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd96259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd96259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54942/0 (socket says 172.21.15.145:54942) 2023-08-25T04:35:56.460 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.675+0000 7fd96259c700 1 -- 172.21.15.145:0/3644460860 learned_addr learned my addr 172.21.15.145:0/3644460860 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd96259c700 1 -- 172.21.15.145:0/3644460860 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954009550 con 0x7fd9641061b0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd96259c700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd954006af0 tx=0x7fd954011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd954003a40 con 0x7fd9641061b0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd954003ba0 con 0x7fd9641061b0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96407a230 con 0x7fd9641061b0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd95401a700 con 0x7fd9641061b0 2023-08-25T04:35:56.461 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.676+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96407a650 con 0x7fd9641061b0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.677+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7fd954024430 con 0x7fd9641061b0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.677+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964061f50 con 0x7fd9641061b0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.678+0000 7fd9537fe700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 0x7fd94c03a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.678+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd95404be60 con 0x7fd9641061b0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.678+0000 7fd961d9b700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 0x7fd94c03a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.678+0000 7fd961d9b700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 0x7fd94c03a5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd958006fd0 tx=0x7fd958006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:56.462 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.683+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd9540189a0 con 0x7fd9641061b0 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.959+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd96410a5a0 con 0x7fd9641061b0 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.960+0000 7fd9537fe700 1 -- 172.21.15.145:0/3644460860 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd9540187c0 con 0x7fd9641061b0 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 msgr2=0x7fd94c03a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 0x7fd94c03a5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd958006fd0 tx=0x7fd958006e40 comp rx=0 tx=0).stop 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 msgr2=0x7fd96407bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd954006af0 tx=0x7fd954011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:56.463 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 shutdown_connections 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fd94c038110 0x7fd94c03a5d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 --2- 172.21.15.145:0/3644460860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9641061b0 0x7fd96407bbe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.962+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 >> 172.21.15.145:0/3644460860 conn(0x7fd964101750 msgr2=0x7fd964102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.963+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 shutdown_connections 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:55.963+0000 7fd968909700 1 -- 172.21.15.145:0/3644460860 wait complete. 2023-08-25T04:35:56.464 INFO:teuthology.orchestra.run.smithi145.stdout:mgr is available 2023-08-25T04:35:57.558 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:57 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/901190724' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-25T04:35:57.625 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:57.625 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [global] 2023-08-25T04:35:57.625 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout fsid = b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:35:57.625 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-25T04:35:57.625 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout [osd] 2023-08-25T04:35:57.626 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.935+0000 7ff946099700 1 Processor -- start 2023-08-25T04:35:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.935+0000 7ff946099700 1 -- start start 2023-08-25T04:35:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.935+0000 7ff946099700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.935+0000 7ff946099700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94007a860 con 0x7ff94007be30 2023-08-25T04:35:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54958/0 (socket says 172.21.15.145:54958) 2023-08-25T04:35:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93f7fe700 1 -- 172.21.15.145:0/545638125 learned_addr learned my addr 172.21.15.145:0/545638125 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:57.629 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93f7fe700 1 -- 172.21.15.145:0/545638125 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94007a9a0 con 0x7ff94007be30 2023-08-25T04:35:57.629 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93f7fe700 1 --2- 172.21.15.145:0/545638125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff9300098a0 tx=0x7ff930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad93cd44d258db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:57.629 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.936+0000 7ff93e7fc700 1 -- 172.21.15.145:0/545638125 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff930004160 con 0x7ff94007be30 2023-08-25T04:35:57.629 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff93e7fc700 1 -- 172.21.15.145:0/545638125 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9300042c0 con 0x7ff94007be30 2023-08-25T04:35:57.629 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff93e7fc700 1 -- 172.21.15.145:0/545638125 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff930011a00 con 0x7ff94007be30 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 -- 172.21.15.145:0/545638125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 msgr2=0x7ff94007a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 --2- 172.21.15.145:0/545638125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff9300098a0 tx=0x7ff930009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 -- 172.21.15.145:0/545638125 shutdown_connections 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 --2- 172.21.15.145:0/545638125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94007a290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 -- 172.21.15.145:0/545638125 >> 172.21.15.145:0/545638125 conn(0x7ff940101750 msgr2=0x7ff940103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 -- 172.21.15.145:0/545638125 shutdown_connections 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.937+0000 7ff946099700 1 -- 172.21.15.145:0/545638125 wait complete. 2023-08-25T04:35:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.938+0000 7ff946099700 1 Processor -- start 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.938+0000 7ff946099700 1 -- start start 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff946099700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff946099700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019c5f0 con 0x7ff94007be30 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54962/0 (socket says 172.21.15.145:54962) 2023-08-25T04:35:57.631 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff93f7fe700 1 -- 172.21.15.145:0/901190724 learned_addr learned my addr 172.21.15.145:0/901190724 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.939+0000 7ff93f7fe700 1 -- 172.21.15.145:0/901190724 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930009550 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff93f7fe700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff930009fb0 tx=0x7ff930011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff930003930 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff930003a90 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9300195e0 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94019c7f0 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.940+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94019cc10 con 0x7ff94007be30 2023-08-25T04:35:57.632 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.941+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7ff93001a9d0 con 0x7ff94007be30 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.941+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94010c580 con 0x7ff94007be30 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.941+0000 7ff93cff9700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 0x7ff92803a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.941+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff93004bcb0 con 0x7ff94007be30 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.942+0000 7ff93effd700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 0x7ff92803a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.942+0000 7ff93effd700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 0x7ff92803a580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff934006fd0 tx=0x7ff934006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:56.947+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff930003c00 con 0x7ff94007be30 2023-08-25T04:35:57.633 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.143+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff94010c790 con 0x7ff94007be30 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.144+0000 7ff93cff9700 1 -- 172.21.15.145:0/901190724 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff930023b40 con 0x7ff94007be30 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 msgr2=0x7ff92803a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 0x7ff92803a580 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff934006fd0 tx=0x7ff934006e40 comp rx=0 tx=0).stop 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 msgr2=0x7ff94019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff930009fb0 tx=0x7ff930011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 shutdown_connections 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7ff9280380c0 0x7ff92803a580 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:57.634 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 --2- 172.21.15.145:0/901190724 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff94007be30 0x7ff94019c0b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:57.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 >> 172.21.15.145:0/901190724 conn(0x7ff940101750 msgr2=0x7ff940102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:57.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 shutdown_connections 2023-08-25T04:35:57.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:57.147+0000 7ff946099700 1 -- 172.21.15.145:0/901190724 wait complete. 2023-08-25T04:35:57.635 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling cephadm module... 2023-08-25T04:35:58.738 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1527828770' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-08-25T04:35:58.739 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setuser ceph since I am not root 2023-08-25T04:35:58.739 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setgroup ceph since I am not root 2023-08-25T04:35:58.739 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:58.736+0000 7f7ef0d00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:35:58.899 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.156+0000 7f3fbbf72700 1 Processor -- start 2023-08-25T04:35:58.899 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.156+0000 7f3fbbf72700 1 -- start start 2023-08-25T04:35:58.899 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fbbf72700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:58.899 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fbbf72700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4106ba0 con 0x7f3fb41061b0 2023-08-25T04:35:58.899 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fb9d0e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fb9d0e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54976/0 (socket says 172.21.15.145:54976) 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fb9d0e700 1 -- 172.21.15.145:0/2357451406 learned_addr learned my addr 172.21.15.145:0/2357451406 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.157+0000 7f3fb9d0e700 1 -- 172.21.15.145:0/2357451406 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb41073b0 con 0x7f3fb41061b0 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fb9d0e700 1 --2- 172.21.15.145:0/2357451406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098a0 tx=0x7f3fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b21dac28b4171 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fb8d0c700 1 -- 172.21.15.145:0/2357451406 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0004160 con 0x7f3fb41061b0 2023-08-25T04:35:58.900 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fb8d0c700 1 -- 172.21.15.145:0/2357451406 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fb00042c0 con 0x7f3fb41061b0 2023-08-25T04:35:58.901 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fb8d0c700 1 -- 172.21.15.145:0/2357451406 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0011a00 con 0x7f3fb41061b0 2023-08-25T04:35:58.901 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fbbf72700 1 -- 172.21.15.145:0/2357451406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 msgr2=0x7f3fb41065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.158+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/2357451406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098a0 tx=0x7f3fb0009bb0 comp rx=0 tx=0).stop 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.159+0000 7f3fbbf72700 1 -- 172.21.15.145:0/2357451406 shutdown_connections 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.159+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/2357451406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb41065d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.159+0000 7f3fbbf72700 1 -- 172.21.15.145:0/2357451406 >> 172.21.15.145:0/2357451406 conn(0x7f3fb4101750 msgr2=0x7f3fb4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.159+0000 7f3fbbf72700 1 -- 172.21.15.145:0/2357451406 shutdown_connections 2023-08-25T04:35:58.904 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.159+0000 7f3fbbf72700 1 -- 172.21.15.145:0/2357451406 wait complete. 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.160+0000 7f3fbbf72700 1 Processor -- start 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.160+0000 7f3fbbf72700 1 -- start start 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fbbf72700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fbbf72700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb419a3e0 con 0x7f3fb41061b0 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fb9d0e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fb9d0e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54988/0 (socket says 172.21.15.145:54988) 2023-08-25T04:35:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fb9d0e700 1 -- 172.21.15.145:0/1527828770 learned_addr learned my addr 172.21.15.145:0/1527828770 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.161+0000 7f3fb9d0e700 1 -- 172.21.15.145:0/1527828770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0009550 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3fb9d0e700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006af0 tx=0x7f3fb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0003a40 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fb0003ba0 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb419a5e0 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb001a700 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.162+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb419aa00 con 0x7f3fb41061b0 2023-08-25T04:35:58.906 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.163+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f3fb0024430 con 0x7f3fb41061b0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.163+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb4061f50 con 0x7f3fb41061b0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.163+0000 7f3faaffd700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 0x7f3fa003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.163+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fb004be60 con 0x7f3fb41061b0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.164+0000 7f3fb950d700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 0x7f3fa003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.164+0000 7f3fb950d700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 0x7f3fa003a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006fd0 tx=0x7f3fa4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.169+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3fb00189a0 con 0x7f3fb41061b0 2023-08-25T04:35:58.907 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.416+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f3fb419d780 con 0x7f3fb41061b0 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.469+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f3fb00246e0 con 0x7f3fb41061b0 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.470+0000 7f3faaffd700 1 -- 172.21.15.145:0/1527828770 <== mon.0 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f3fb0017d00 con 0x7f3fb41061b0 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 msgr2=0x7f3fa003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 0x7f3fa003a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006fd0 tx=0x7f3fa4006e40 comp rx=0 tx=0).stop 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 msgr2=0x7f3fb4199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006af0 tx=0x7f3fb0011ad0 comp rx=0 tx=0).stop 2023-08-25T04:35:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 shutdown_connections 2023-08-25T04:35:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f3fa0038110 0x7f3fa003a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 --2- 172.21.15.145:0/1527828770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fb41061b0 0x7f3fb4199ea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:35:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 >> 172.21.15.145:0/1527828770 conn(0x7f3fb4101750 msgr2=0x7f3fb4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:35:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.473+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 shutdown_connections 2023-08-25T04:35:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:58.474+0000 7f3fbbf72700 1 -- 172.21.15.145:0/1527828770 wait complete. 2023-08-25T04:35:59.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:58.841+0000 7f7ef0d00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:35:59.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:58.967+0000 7f7ef0d00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:35:59.475 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:59.093+0000 7f7ef0d00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:35:59.475 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:59.334+0000 7f7ef0d00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:35:59.724 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1527828770' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-08-25T04:35:59.725 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-mon[97681]: mgrmap e5: x(active, since 8s) 2023-08-25T04:35:59.989 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:59.820+0000 7f7ef0d00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:36:00.190 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:36:00.190 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-08-25T04:36:00.191 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T04:36:00.192 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-25T04:36:00.192 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-25T04:36:00.192 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:36:00.192 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.422+0000 7fb0a3534700 1 Processor -- start 2023-08-25T04:36:00.193 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.422+0000 7fb0a3534700 1 -- start start 2023-08-25T04:36:00.193 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.422+0000 7fb0a3534700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:00.193 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.422+0000 7fb0a3534700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c07a840 con 0x7fb09c07be10 2023-08-25T04:36:00.194 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.422+0000 7fb0a12d0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:00.194 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.423+0000 7fb0a12d0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:55018/0 (socket says 172.21.15.145:55018) 2023-08-25T04:36:00.194 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.423+0000 7fb0a12d0700 1 -- 172.21.15.145:0/3376708023 learned_addr learned my addr 172.21.15.145:0/3376708023 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:00.194 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.423+0000 7fb0a12d0700 1 -- 172.21.15.145:0/3376708023 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c07a980 con 0x7fb09c07be10 2023-08-25T04:36:00.194 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.423+0000 7fb0a12d0700 1 --2- 172.21.15.145:0/3376708023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098a0 tx=0x7fb098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5127f664e8a4420a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:00.195 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.424+0000 7fb093fff700 1 -- 172.21.15.145:0/3376708023 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098004160 con 0x7fb09c07be10 2023-08-25T04:36:00.195 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.424+0000 7fb093fff700 1 -- 172.21.15.145:0/3376708023 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0980042c0 con 0x7fb09c07be10 2023-08-25T04:36:00.195 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.424+0000 7fb093fff700 1 -- 172.21.15.145:0/3376708023 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098011a00 con 0x7fb09c07be10 2023-08-25T04:36:00.195 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.424+0000 7fb0a3534700 1 -- 172.21.15.145:0/3376708023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 msgr2=0x7fb09c07a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:00.195 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.424+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3376708023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098a0 tx=0x7fb098009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.425+0000 7fb0a3534700 1 -- 172.21.15.145:0/3376708023 shutdown_connections 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.425+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3376708023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c07a270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.425+0000 7fb0a3534700 1 -- 172.21.15.145:0/3376708023 >> 172.21.15.145:0/3376708023 conn(0x7fb09c101750 msgr2=0x7fb09c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.425+0000 7fb0a3534700 1 -- 172.21.15.145:0/3376708023 shutdown_connections 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.425+0000 7fb0a3534700 1 -- 172.21.15.145:0/3376708023 wait complete. 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.426+0000 7fb0a3534700 1 Processor -- start 2023-08-25T04:36:00.196 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.426+0000 7fb0a3534700 1 -- start start 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a3534700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a3534700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c19c580 con 0x7fb09c07be10 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a12d0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a12d0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:55022/0 (socket says 172.21.15.145:55022) 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a12d0700 1 -- 172.21.15.145:0/3029353618 learned_addr learned my addr 172.21.15.145:0/3029353618 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.427+0000 7fb0a12d0700 1 -- 172.21.15.145:0/3029353618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098009550 con 0x7fb09c07be10 2023-08-25T04:36:00.197 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0a12d0700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb098009fb0 tx=0x7fb098011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098003930 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb098003a90 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098019610 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c19c780 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.428+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c19cba0 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.429+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fb09801a9d0 con 0x7fb09c07be10 2023-08-25T04:36:00.198 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.429+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c10c5a0 con 0x7fb09c07be10 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.430+0000 7fb0927fc700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 0x7fb08803a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.430+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb09804bd90 con 0x7fb09c07be10 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.430+0000 7fb0a0acf700 1 -- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 msgr2=0x7fb08803a580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.430+0000 7fb0a0acf700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 0x7fb08803a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.435+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb098003c00 con 0x7fb09c07be10 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.631+0000 7fb0a0acf700 1 -- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 msgr2=0x7fb08803a580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:00.199 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.631+0000 7fb0a0acf700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 0x7fb08803a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.697+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb09c10c7b0 con 0x7fb09c07be10 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.698+0000 7fb0927fc700 1 -- 172.21.15.145:0/3029353618 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb098018350 con 0x7fb09c07be10 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 msgr2=0x7fb08803a580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 0x7fb08803a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 msgr2=0x7fb09c19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb098009fb0 tx=0x7fb098011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.200 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 shutdown_connections 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7fb0880380c0 0x7fb08803a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 --2- 172.21.15.145:0/3029353618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb09c07be10 0x7fb09c19c040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 >> 172.21.15.145:0/3029353618 conn(0x7fb09c101750 msgr2=0x7fb09c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 shutdown_connections 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:35:59.701+0000 7fb0a3534700 1 -- 172.21.15.145:0/3029353618 wait complete. 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for the mgr to restart... 2023-08-25T04:36:00.201 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mgr epoch 5... 2023-08-25T04:36:00.245 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:35:59 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:35:59.987+0000 7f7ef0d00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:36:00.245 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:00.107+0000 7f7ef0d00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:36:00.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:00 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3029353618' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-25T04:36:00.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:00.242+0000 7f7ef0d00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:36:01.077 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:00.796+0000 7f7ef0d00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:36:02.077 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:01 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:01.640+0000 7f7ef0d00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:36:02.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:01 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:01.746+0000 7f7ef0d00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:36:03.036 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:02 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:02.693+0000 7f7ef0d00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:36:03.037 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:02 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:02.812+0000 7f7ef0d00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:36:03.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:03.035+0000 7f7ef0d00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:36:03.670 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:03.394+0000 7f7ef0d00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:36:04.077 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:03.669+0000 7f7ef0d00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:36:04.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:03.858+0000 7f7ef0d00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:36:04.577 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:04.198+0000 7f7ef0d00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:36:05.532 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:05 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:05.117+0000 7f7ef0d00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:36:05.532 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:05 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:05.288+0000 7f7ef0d00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:36:05.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:05 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:05.531+0000 7f7ef0d00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:36:05.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:05 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:05.673+0000 7f7ef0d00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:36:06.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:05 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:05.942+0000 7f7ef0d00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:36:07.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:06 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:06.997+0000 7f7ef0d00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:36:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: Active manager daemon x restarted 2023-08-25T04:36:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: Activating manager daemon x 2023-08-25T04:36:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: osdmap e2: 0 total, 0 up, 0 in 2023-08-25T04:36:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: mgrmap e6: x(active, starting, since 0.00393293s) 2023-08-25T04:36:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: Manager daemon x is now available 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:07.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:07.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:07.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:07.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:07 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: Found migration_current of "None". Setting to last migration. 2023-08-25T04:36:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:36:08.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:36:08.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:08.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:08.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:07] ENGINE Bus STARTING 2023-08-25T04:36:08.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:08.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:08 smithi145 ceph-mon[97681]: mgrmap e7: x(active, since 1.00771s) 2023-08-25T04:36:08.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:36:08.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-08-25T04:36:08.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-25T04:36:08.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:36:08.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.914+0000 7f017168a700 1 Processor -- start 2023-08-25T04:36:08.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.914+0000 7f017168a700 1 -- start start 2023-08-25T04:36:08.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.914+0000 7f017168a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:08.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.914+0000 7f017168a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c07a840 con 0x7f016c07be10 2023-08-25T04:36:08.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.915+0000 7f016affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:08.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.915+0000 7f016affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:55032/0 (socket says 172.21.15.145:55032) 2023-08-25T04:36:08.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.915+0000 7f016affd700 1 -- 172.21.15.145:0/2013565919 learned_addr learned my addr 172.21.15.145:0/2013565919 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.915+0000 7f016affd700 1 -- 172.21.15.145:0/2013565919 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c07a980 con 0x7f016c07be10 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.915+0000 7f016affd700 1 --2- 172.21.15.145:0/2013565919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f015c0098a0 tx=0x7f015c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4298e8bd7fe07b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f0169ffb700 1 -- 172.21.15.145:0/2013565919 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f015c004160 con 0x7f016c07be10 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f0169ffb700 1 -- 172.21.15.145:0/2013565919 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f015c0042c0 con 0x7f016c07be10 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f0169ffb700 1 -- 172.21.15.145:0/2013565919 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f015c011a00 con 0x7f016c07be10 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f017168a700 1 -- 172.21.15.145:0/2013565919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 msgr2=0x7f016c07a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:08.496 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f017168a700 1 --2- 172.21.15.145:0/2013565919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f015c0098a0 tx=0x7f015c009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f017168a700 1 -- 172.21.15.145:0/2013565919 shutdown_connections 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f017168a700 1 --2- 172.21.15.145:0/2013565919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c07a270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.916+0000 7f017168a700 1 -- 172.21.15.145:0/2013565919 >> 172.21.15.145:0/2013565919 conn(0x7f016c101750 msgr2=0x7f016c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.917+0000 7f017168a700 1 -- 172.21.15.145:0/2013565919 shutdown_connections 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.917+0000 7f017168a700 1 -- 172.21.15.145:0/2013565919 wait complete. 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f017168a700 1 Processor -- start 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f017168a700 1 -- start start 2023-08-25T04:36:08.497 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f017168a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f017168a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c19c580 con 0x7f016c07be10 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f016affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f016affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:55044/0 (socket says 172.21.15.145:55044) 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.918+0000 7f016affd700 1 -- 172.21.15.145:0/1450141238 learned_addr learned my addr 172.21.15.145:0/1450141238 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f016affd700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c009550 con 0x7f016c07be10 2023-08-25T04:36:08.498 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f016affd700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f015c006af0 tx=0x7f015c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:08.499 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f015c003a40 con 0x7f016c07be10 2023-08-25T04:36:08.499 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f015c003ba0 con 0x7f016c07be10 2023-08-25T04:36:08.499 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f015c01a6e0 con 0x7f016c07be10 2023-08-25T04:36:08.499 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f017168a700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c19c780 con 0x7f016c07be10 2023-08-25T04:36:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.919+0000 7f017168a700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c19cba0 con 0x7f016c07be10 2023-08-25T04:36:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.920+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f015c024a60 con 0x7f016c07be10 2023-08-25T04:36:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.921+0000 7f0153fff700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.921+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f015403ad30 con 0x7f0154038160 2023-08-25T04:36:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.921+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f015c04bf70 con 0x7f016c07be10 2023-08-25T04:36:08.501 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.921+0000 7f016a7fc700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:08.501 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:00.921+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:36:08.501 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:01.122+0000 7f016a7fc700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:08.501 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:01.122+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:01.523+0000 7f016a7fc700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:01.523+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:02.324+0000 7f016a7fc700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:02.324+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:03.926+0000 7f016a7fc700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/2347349168 2023-08-25T04:36:08.502 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:03.926+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T04:36:08.503 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:07.001+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mgrmap(e 6) v1 ==== 44902+0+0 (secure 0 0 0) 0x7f015c024610 con 0x7f016c07be10 2023-08-25T04:36:08.503 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:07.002+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 msgr2=0x7f015403a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:36:08.503 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:07.002+0000 7f0153fff700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.503 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.004+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7f015c04d500 con 0x7f016c07be10 2023-08-25T04:36:08.503 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.004+0000 7f0153fff700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 0x7f015403df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:08.504 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.004+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f015403ad30 con 0x7f015403bba0 2023-08-25T04:36:08.504 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.005+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 0x7f015403df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:08.504 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.006+0000 7f016a7fc700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 0x7f015403df90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0160003a10 tx=0x7f01600092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:08.504 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.007+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f015403ad30 con 0x7f015403bba0 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.017+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f014c002600 con 0x7f015403bba0 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.017+0000 7f0153fff700 1 -- 172.21.15.145:0/1450141238 <== mgr.14118 v2:172.21.15.145:6800/3601102351 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f014c002600 con 0x7f015403bba0 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 msgr2=0x7f015403df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 0x7f015403df90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0160003a10 tx=0x7f01600092b0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 msgr2=0x7f016c19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f015c006af0 tx=0x7f015c011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.505 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 shutdown_connections 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f015403bba0 0x7f015403df90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f016c07be10 0x7f016c19c040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 --2- 172.21.15.145:0/1450141238 >> [v2:172.21.15.145:6800/2347349168,v1:172.21.15.145:6801/2347349168] conn(0x7f0154038160 0x7f015403a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 >> 172.21.15.145:0/1450141238 conn(0x7f016c101750 msgr2=0x7f016c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 shutdown_connections 2023-08-25T04:36:08.506 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:08.018+0000 7f0151ffb700 1 -- 172.21.15.145:0/1450141238 wait complete. 2023-08-25T04:36:08.507 INFO:teuthology.orchestra.run.smithi145.stdout:mgr epoch 5 is available 2023-08-25T04:36:08.507 INFO:teuthology.orchestra.run.smithi145.stdout:Setting orchestrator backend to cephadm... 2023-08-25T04:36:09.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:09 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:07] ENGINE Serving on https://172.21.15.145:7150 2023-08-25T04:36:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:09 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:07] ENGINE Serving on http://172.21.15.145:8765 2023-08-25T04:36:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:09 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:07] ENGINE Bus STARTED 2023-08-25T04:36:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:09 smithi145 ceph-mon[97681]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-25T04:36:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:09 smithi145 ceph-mon[97681]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-25T04:36:09.677 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.029+0000 7f30dbef0700 1 Processor -- start 2023-08-25T04:36:09.677 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30dbef0700 1 -- start start 2023-08-25T04:36:09.677 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30dbef0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:09.678 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30dbef0700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d407a860 con 0x7f30d407be30 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30d9c8c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30d9c8c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46586/0 (socket says 172.21.15.145:46586) 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.030+0000 7f30d9c8c700 1 -- 172.21.15.145:0/1243572246 learned_addr learned my addr 172.21.15.145:0/1243572246 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.031+0000 7f30d9c8c700 1 -- 172.21.15.145:0/1243572246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d407a9a0 con 0x7f30d407be30 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.031+0000 7f30d9c8c700 1 --2- 172.21.15.145:0/1243572246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30d00098a0 tx=0x7f30d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ee4cd19034248e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.031+0000 7f30d8c8a700 1 -- 172.21.15.145:0/1243572246 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0004160 con 0x7f30d407be30 2023-08-25T04:36:09.679 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.031+0000 7f30d8c8a700 1 -- 172.21.15.145:0/1243572246 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d00042c0 con 0x7f30d407be30 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.031+0000 7f30d8c8a700 1 -- 172.21.15.145:0/1243572246 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0011a00 con 0x7f30d407be30 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 -- 172.21.15.145:0/1243572246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 msgr2=0x7f30d407a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1243572246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30d00098a0 tx=0x7f30d0009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 -- 172.21.15.145:0/1243572246 shutdown_connections 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1243572246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d407a290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 -- 172.21.15.145:0/1243572246 >> 172.21.15.145:0/1243572246 conn(0x7f30d4101750 msgr2=0x7f30d4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:09.680 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 -- 172.21.15.145:0/1243572246 shutdown_connections 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.032+0000 7f30dbef0700 1 -- 172.21.15.145:0/1243572246 wait complete. 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30dbef0700 1 Processor -- start 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30dbef0700 1 -- start start 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30dbef0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30dbef0700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d419c5d0 con 0x7f30d407be30 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30d9c8c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:09.681 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30d9c8c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46602/0 (socket says 172.21.15.145:46602) 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.033+0000 7f30d9c8c700 1 -- 172.21.15.145:0/1543491453 learned_addr learned my addr 172.21.15.145:0/1543491453 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30d9c8c700 1 -- 172.21.15.145:0/1543491453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0009550 con 0x7f30d407be30 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30d9c8c700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006af0 tx=0x7f30d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0003a40 con 0x7f30d407be30 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d0003ba0 con 0x7f30d407be30 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d001a700 con 0x7f30d407be30 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d419c7d0 con 0x7f30d407be30 2023-08-25T04:36:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.034+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d419cbf0 con 0x7f30d407be30 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.036+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7f30d0024ad0 con 0x7f30d407be30 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.036+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d4061f50 con 0x7f30d407be30 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.036+0000 7f30caffd700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 0x7f30c003a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.036+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f30d004bdc0 con 0x7f30d407be30 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.036+0000 7f30d948b700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 0x7f30c003a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.037+0000 7f30d948b700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 0x7f30c003a4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:09.683 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.042+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f30d0018b80 con 0x7f30d407be30 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.057+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f30d002d910 con 0x7f30d407be30 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.254+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d4103a60 con 0x7f30c0037fe0 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.260+0000 7f30caffd700 1 -- 172.21.15.145:0/1543491453 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f30d4103a60 con 0x7f30c0037fe0 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 msgr2=0x7f30c003a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 0x7f30c003a4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 msgr2=0x7f30d419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006af0 tx=0x7f30d0011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 shutdown_connections 2023-08-25T04:36:09.685 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f30c0037fe0 0x7f30c003a4a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:09.685 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 --2- 172.21.15.145:0/1543491453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30d407be30 0x7f30d419c090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:09.685 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 >> 172.21.15.145:0/1543491453 conn(0x7f30d4101750 msgr2=0x7f30d4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:09.685 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 shutdown_connections 2023-08-25T04:36:09.685 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:09.263+0000 7f30dbef0700 1 -- 172.21.15.145:0/1543491453 wait complete. 2023-08-25T04:36:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:10 smithi145 ceph-mon[97681]: mgrmap e8: x(active, since 2s) 2023-08-25T04:36:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:10 smithi145 ceph-mon[97681]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:10 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:10 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:10.854 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout value unchanged 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.217+0000 7f485420b700 1 Processor -- start 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.217+0000 7f485420b700 1 -- start start 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f485420b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f485420b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c106ba0 con 0x7f484c1061b0 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f4851fa7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:10.855 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f4851fa7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46606/0 (socket says 172.21.15.145:46606) 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f4851fa7700 1 -- 172.21.15.145:0/456909885 learned_addr learned my addr 172.21.15.145:0/456909885 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.218+0000 7f4851fa7700 1 -- 172.21.15.145:0/456909885 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c1073b0 con 0x7f484c1061b0 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f4851fa7700 1 --2- 172.21.15.145:0/456909885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48480098a0 tx=0x7f4848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c426c4dc89f516ee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f4850fa5700 1 -- 172.21.15.145:0/456909885 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4848004160 con 0x7f484c1061b0 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f4850fa5700 1 -- 172.21.15.145:0/456909885 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48480042c0 con 0x7f484c1061b0 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f4850fa5700 1 -- 172.21.15.145:0/456909885 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4848011a00 con 0x7f484c1061b0 2023-08-25T04:36:10.856 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f485420b700 1 -- 172.21.15.145:0/456909885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 msgr2=0x7f484c1065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.219+0000 7f485420b700 1 --2- 172.21.15.145:0/456909885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48480098a0 tx=0x7f4848009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.220+0000 7f485420b700 1 -- 172.21.15.145:0/456909885 shutdown_connections 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.220+0000 7f485420b700 1 --2- 172.21.15.145:0/456909885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c1065d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.220+0000 7f485420b700 1 -- 172.21.15.145:0/456909885 >> 172.21.15.145:0/456909885 conn(0x7f484c101750 msgr2=0x7f484c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.220+0000 7f485420b700 1 -- 172.21.15.145:0/456909885 shutdown_connections 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.220+0000 7f485420b700 1 -- 172.21.15.145:0/456909885 wait complete. 2023-08-25T04:36:10.857 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f485420b700 1 Processor -- start 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f485420b700 1 -- start start 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f485420b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f485420b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c07c120 con 0x7f484c1061b0 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f4851fa7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f4851fa7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46610/0 (socket says 172.21.15.145:46610) 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.221+0000 7f4851fa7700 1 -- 172.21.15.145:0/673166038 learned_addr learned my addr 172.21.15.145:0/673166038 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f4851fa7700 1 -- 172.21.15.145:0/673166038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4848009550 con 0x7f484c1061b0 2023-08-25T04:36:10.858 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f4851fa7700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4848006af0 tx=0x7f4848011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:10.859 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4848003a40 con 0x7f484c1061b0 2023-08-25T04:36:10.859 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4848003ba0 con 0x7f484c1061b0 2023-08-25T04:36:10.860 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c07a230 con 0x7f484c1061b0 2023-08-25T04:36:10.860 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f484801a700 con 0x7f484c1061b0 2023-08-25T04:36:10.860 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.222+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c07a650 con 0x7f484c1061b0 2023-08-25T04:36:10.860 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.223+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f4848024ad0 con 0x7f484c1061b0 2023-08-25T04:36:10.861 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.223+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c061f50 con 0x7f484c1061b0 2023-08-25T04:36:10.861 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.224+0000 7f4842ffd700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 0x7f483803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:10.861 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.224+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f484804bfd0 con 0x7f484c1061b0 2023-08-25T04:36:10.861 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.224+0000 7f48517a6700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 0x7f483803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:10.862 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.225+0000 7f48517a6700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 0x7f483803a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f483c006fd0 tx=0x7f483c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:10.862 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.229+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4848018b80 con 0x7f484c1061b0 2023-08-25T04:36:10.862 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.429+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f484c07ad60 con 0x7f4838038160 2023-08-25T04:36:10.863 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.430+0000 7f4842ffd700 1 -- 172.21.15.145:0/673166038 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f484c07ad60 con 0x7f4838038160 2023-08-25T04:36:10.863 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 msgr2=0x7f483803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:10.863 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 0x7f483803a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f483c006fd0 tx=0x7f483c006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:10.864 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 msgr2=0x7f484c07bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:10.864 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4848006af0 tx=0x7f4848011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:10.864 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 shutdown_connections 2023-08-25T04:36:10.864 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f4838038160 0x7f483803a620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:10.865 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 --2- 172.21.15.145:0/673166038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484c1061b0 0x7f484c07bbe0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:10.865 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 >> 172.21.15.145:0/673166038 conn(0x7f484c101750 msgr2=0x7f484c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:10.865 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 shutdown_connections 2023-08-25T04:36:10.865 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:10.433+0000 7f485420b700 1 -- 172.21.15.145:0/673166038 wait complete. 2023-08-25T04:36:10.865 INFO:teuthology.orchestra.run.smithi145.stdout:Generating ssh key... 2023-08-25T04:36:11.114 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:11 smithi145 ceph-mon[97681]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:12.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: Generating public/private rsa key pair. 2023-08-25T04:36:12.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: Your identification has been saved in /tmp/tmp73vl7qxf/key. 2023-08-25T04:36:12.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: Your public key has been saved in /tmp/tmp73vl7qxf/key.pub. 2023-08-25T04:36:12.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: The key fingerprint is: 2023-08-25T04:36:12.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: SHA256:eZrp9U0bJHqoAZ8b2eXMKpghEANVx77FwkUkM2GUJP0 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: The key's randomart image is: 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: +---[RSA 3072]----+ 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: |.o..o=X=o | 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | o +++. | 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | o o + | 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | . + E. | 2023-08-25T04:36:12.579 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | . .+S . o . | 2023-08-25T04:36:12.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | . oo O B o | 2023-08-25T04:36:12.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | . +X = = o | 2023-08-25T04:36:12.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | o..* + o o | 2023-08-25T04:36:12.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: | +.. . o | 2023-08-25T04:36:12.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: +----[SHA256]-----+ 2023-08-25T04:36:12.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.382+0000 7f57d28f5700 1 Processor -- start 2023-08-25T04:36:12.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.382+0000 7f57d28f5700 1 -- start start 2023-08-25T04:36:12.635 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.382+0000 7f57d28f5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:12.636 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.382+0000 7f57d28f5700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc07a820 con 0x7f57cc07bdf0 2023-08-25T04:36:12.636 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.382+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:12.636 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46626/0 (socket says 172.21.15.145:46626) 2023-08-25T04:36:12.636 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57cbfff700 1 -- 172.21.15.145:0/2370413232 learned_addr learned my addr 172.21.15.145:0/2370413232 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:12.637 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57cbfff700 1 -- 172.21.15.145:0/2370413232 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc07a960 con 0x7f57cc07bdf0 2023-08-25T04:36:12.637 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57cbfff700 1 --2- 172.21.15.145:0/2370413232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0098a0 tx=0x7f57bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1162c2e7904920e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:12.637 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57caffd700 1 -- 172.21.15.145:0/2370413232 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57bc004160 con 0x7f57cc07bdf0 2023-08-25T04:36:12.637 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57caffd700 1 -- 172.21.15.145:0/2370413232 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57bc0042c0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.638 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.383+0000 7f57caffd700 1 -- 172.21.15.145:0/2370413232 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57bc011a00 con 0x7f57cc07bdf0 2023-08-25T04:36:12.638 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 -- 172.21.15.145:0/2370413232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 msgr2=0x7f57cc07a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:12.638 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2370413232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0098a0 tx=0x7f57bc009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:12.639 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 -- 172.21.15.145:0/2370413232 shutdown_connections 2023-08-25T04:36:12.639 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2370413232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc07a250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:12.639 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 -- 172.21.15.145:0/2370413232 >> 172.21.15.145:0/2370413232 conn(0x7f57cc101750 msgr2=0x7f57cc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:12.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 -- 172.21.15.145:0/2370413232 shutdown_connections 2023-08-25T04:36:12.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.384+0000 7f57d28f5700 1 -- 172.21.15.145:0/2370413232 wait complete. 2023-08-25T04:36:12.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.385+0000 7f57d28f5700 1 Processor -- start 2023-08-25T04:36:12.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.385+0000 7f57d28f5700 1 -- start start 2023-08-25T04:36:12.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57d28f5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:12.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57d28f5700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19c5a0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:12.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46630/0 (socket says 172.21.15.145:46630) 2023-08-25T04:36:12.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57cbfff700 1 -- 172.21.15.145:0/2983970200 learned_addr learned my addr 172.21.15.145:0/2983970200 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:12.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57cbfff700 1 -- 172.21.15.145:0/2983970200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009550 con 0x7f57cc07bdf0 2023-08-25T04:36:12.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57cbfff700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006af0 tx=0x7f57bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.386+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57bc003a40 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.387+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57bc003ba0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.387+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc19c7a0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.387+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57bc01a6e0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.387+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc19cbc0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.388+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f57bc024430 con 0x7f57cc07bdf0 2023-08-25T04:36:12.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.388+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc061f50 con 0x7f57cc07bdf0 2023-08-25T04:36:12.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.388+0000 7f57c97fa700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 0x7f57b403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.388+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57bc04bfe0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.388+0000 7f57cb7fe700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 0x7f57b403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.389+0000 7f57cb7fe700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 0x7f57b403a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.393+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f57bc0246e0 con 0x7f57cc07bdf0 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:11.590+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f57cc103a70 con 0x7f57b4038160 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.169+0000 7f57c97fa700 1 -- 172.21.15.145:0/2983970200 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f57cc103a70 con 0x7f57b4038160 2023-08-25T04:36:12.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.171+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 msgr2=0x7f57b403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 0x7f57b403a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57c0006fd0 tx=0x7f57c0006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 msgr2=0x7f57cc19c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006af0 tx=0x7f57bc011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 shutdown_connections 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f57b4038160 0x7f57b403a620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 --2- 172.21.15.145:0/2983970200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57cc07bdf0 0x7f57cc19c060 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:12.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 >> 172.21.15.145:0/2983970200 conn(0x7f57cc101750 msgr2=0x7f57cc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:12.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 shutdown_connections 2023-08-25T04:36:12.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:12.172+0000 7f57d28f5700 1 -- 172.21.15.145:0/2983970200 wait complete. 2023-08-25T04:36:13.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:13 smithi145 ceph-mon[97681]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:13.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:13 smithi145 ceph-mon[97681]: Generating ssh key... 2023-08-25T04:36:13.496 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:13 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:13.497 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:13 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:13.812 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:36:13.812 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.159+0000 7f38b2688700 1 Processor -- start 2023-08-25T04:36:13.812 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.159+0000 7f38b2688700 1 -- start start 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.159+0000 7f38b2688700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.159+0000 7f38b2688700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac105a90 con 0x7f38ac1050a0 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38abfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38abfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60674/0 (socket says 172.21.15.145:60674) 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38abfff700 1 -- 172.21.15.145:0/569682887 learned_addr learned my addr 172.21.15.145:0/569682887 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38abfff700 1 -- 172.21.15.145:0/569682887 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac1062a0 con 0x7f38ac1050a0 2023-08-25T04:36:13.813 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38abfff700 1 --2- 172.21.15.145:0/569682887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a8314ae9dafaf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.160+0000 7f38aaffd700 1 -- 172.21.15.145:0/569682887 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c004160 con 0x7f38ac1050a0 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38aaffd700 1 -- 172.21.15.145:0/569682887 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f389c0042c0 con 0x7f38ac1050a0 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38aaffd700 1 -- 172.21.15.145:0/569682887 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c011a00 con 0x7f38ac1050a0 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 -- 172.21.15.145:0/569682887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 msgr2=0x7f38ac1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 --2- 172.21.15.145:0/569682887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:13.814 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 -- 172.21.15.145:0/569682887 shutdown_connections 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 --2- 172.21.15.145:0/569682887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac1054c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 -- 172.21.15.145:0/569682887 >> 172.21.15.145:0/569682887 conn(0x7f38ac100620 msgr2=0x7f38ac102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 -- 172.21.15.145:0/569682887 shutdown_connections 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.161+0000 7f38b2688700 1 -- 172.21.15.145:0/569682887 wait complete. 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.162+0000 7f38b2688700 1 Processor -- start 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.162+0000 7f38b2688700 1 -- start start 2023-08-25T04:36:13.815 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38b2688700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38b2688700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac19a3d0 con 0x7f38ac1050a0 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38abfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38abfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60688/0 (socket says 172.21.15.145:60688) 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38abfff700 1 -- 172.21.15.145:0/3918252461 learned_addr learned my addr 172.21.15.145:0/3918252461 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.163+0000 7f38abfff700 1 -- 172.21.15.145:0/3918252461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009550 con 0x7f38ac1050a0 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.164+0000 7f38abfff700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f389c009fb0 tx=0x7f389c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:13.816 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.166+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c003930 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.166+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac19a5d0 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.166+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f389c003a90 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.166+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c0195e0 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.166+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac19a9f0 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.167+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f389c01a430 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.167+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac061f50 con 0x7f38ac1050a0 2023-08-25T04:36:13.817 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.168+0000 7f38a97fa700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 0x7f389403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.168+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f389c04c040 con 0x7f38ac1050a0 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.168+0000 7f38ab7fe700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 0x7f389403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.169+0000 7f38ab7fe700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 0x7f389403a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.173+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f389c07ee00 con 0x7f38ac1050a0 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.370+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ac102860 con 0x7f3894038110 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.370+0000 7f38a97fa700 1 -- 172.21.15.145:0/3918252461 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f38ac102860 con 0x7f3894038110 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 msgr2=0x7f389403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:13.818 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 0x7f389403a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38a0006fd0 tx=0x7f38a0006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 msgr2=0x7f38ac199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f389c009fb0 tx=0x7f389c011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 shutdown_connections 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3894038110 0x7f389403a5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 --2- 172.21.15.145:0/3918252461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38ac1050a0 0x7f38ac199e90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 >> 172.21.15.145:0/3918252461 conn(0x7f38ac100620 msgr2=0x7f38ac0798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 shutdown_connections 2023-08-25T04:36:13.820 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:13.373+0000 7f38b2688700 1 -- 172.21.15.145:0/3918252461 wait complete. 2023-08-25T04:36:13.820 INFO:teuthology.orchestra.run.smithi145.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-08-25T04:36:13.820 INFO:teuthology.orchestra.run.smithi145.stdout:Adding key to root@localhost authorized_keys... 2023-08-25T04:36:13.820 INFO:teuthology.orchestra.run.smithi145.stdout:Adding host smithi145... 2023-08-25T04:36:14.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:14 smithi145 ceph-mon[97681]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:15.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:15 smithi145 ceph-mon[97681]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "addr": "172.21.15.145", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:16.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:16 smithi145 ceph-mon[97681]: Deploying cephadm binary to smithi145 2023-08-25T04:36:18.097 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout Added host 'smithi145' with addr '172.21.15.145' 2023-08-25T04:36:18.098 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.344+0000 7f052ae44700 1 Processor -- start 2023-08-25T04:36:18.098 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.344+0000 7f052ae44700 1 -- start start 2023-08-25T04:36:18.098 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.344+0000 7f052ae44700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:18.098 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.344+0000 7f052ae44700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052407a860 con 0x7f052407be30 2023-08-25T04:36:18.098 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.344+0000 7f0528be0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:18.099 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f0528be0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60694/0 (socket says 172.21.15.145:60694) 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f0528be0700 1 -- 172.21.15.145:0/85495615 learned_addr learned my addr 172.21.15.145:0/85495615 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f0528be0700 1 -- 172.21.15.145:0/85495615 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052407a9a0 con 0x7f052407be30 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f0528be0700 1 --2- 172.21.15.145:0/85495615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e4bc197461d338 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f05237fe700 1 -- 172.21.15.145:0/85495615 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0518004160 con 0x7f052407be30 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f05237fe700 1 -- 172.21.15.145:0/85495615 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05180042c0 con 0x7f052407be30 2023-08-25T04:36:18.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.345+0000 7f05237fe700 1 -- 172.21.15.145:0/85495615 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0518011a00 con 0x7f052407be30 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 -- 172.21.15.145:0/85495615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 msgr2=0x7f052407a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 --2- 172.21.15.145:0/85495615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 -- 172.21.15.145:0/85495615 shutdown_connections 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 --2- 172.21.15.145:0/85495615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052407a290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 -- 172.21.15.145:0/85495615 >> 172.21.15.145:0/85495615 conn(0x7f0524101750 msgr2=0x7f0524103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 -- 172.21.15.145:0/85495615 shutdown_connections 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.346+0000 7f052ae44700 1 -- 172.21.15.145:0/85495615 wait complete. 2023-08-25T04:36:18.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.347+0000 7f052ae44700 1 Processor -- start 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.347+0000 7f052ae44700 1 -- start start 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.347+0000 7f052ae44700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.347+0000 7f052ae44700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052419c5d0 con 0x7f052407be30 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0528be0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0528be0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60704/0 (socket says 172.21.15.145:60704) 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0528be0700 1 -- 172.21.15.145:0/2019302298 learned_addr learned my addr 172.21.15.145:0/2019302298 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:18.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0528be0700 1 -- 172.21.15.145:0/2019302298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518009550 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0528be0700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0518006af0 tx=0x7f0518011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0518003a40 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.348+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0518003ba0 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.349+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f051801a700 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.349+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052419c7d0 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.349+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052419cbf0 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.350+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f0518024ad0 con 0x7f052407be30 2023-08-25T04:36:18.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.350+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0524061f50 con 0x7f052407be30 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.350+0000 7f0521ffb700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 0x7f051403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.350+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f051804bfd0 con 0x7f052407be30 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.350+0000 7f0523fff700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 0x7f051403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.351+0000 7f0523fff700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 0x7f051403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.355+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0518018b80 con 0x7f052407be30 2023-08-25T04:36:18.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:14.556+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi145", "addr": "172.21.15.145", "target": ["mon-mgr", ""]}) v1 -- 0x7f0524103a60 con 0x7f0514038160 2023-08-25T04:36:18.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.763+0000 7f0521ffb700 1 -- 172.21.15.145:0/2019302298 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0524103a60 con 0x7f0514038160 2023-08-25T04:36:18.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 msgr2=0x7f051403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:18.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 0x7f051403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:18.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 msgr2=0x7f052419c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:18.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0518006af0 tx=0x7f0518011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:18.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 shutdown_connections 2023-08-25T04:36:18.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f0514038160 0x7f051403a620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:18.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 --2- 172.21.15.145:0/2019302298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f052407be30 0x7f052419c090 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:18.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 >> 172.21.15.145:0/2019302298 conn(0x7f0524101750 msgr2=0x7f0524102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:18.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 shutdown_connections 2023-08-25T04:36:18.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:17.766+0000 7f052ae44700 1 -- 172.21.15.145:0/2019302298 wait complete. 2023-08-25T04:36:18.109 INFO:teuthology.orchestra.run.smithi145.stdout:Deploying unmanaged mon service... 2023-08-25T04:36:19.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:18 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:19.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:18 smithi145 ceph-mon[97681]: Added host smithi145 2023-08-25T04:36:19.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:18 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:19.335 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.679+0000 7f273efb7700 1 Processor -- start 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.679+0000 7f273efb7700 1 -- start start 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.679+0000 7f273efb7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.679+0000 7f273efb7700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273807a840 con 0x7f273807be10 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f273cd53700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:19.336 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f273cd53700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60710/0 (socket says 172.21.15.145:60710) 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f273cd53700 1 -- 172.21.15.145:0/2522146475 learned_addr learned my addr 172.21.15.145:0/2522146475 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f273cd53700 1 -- 172.21.15.145:0/2522146475 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273807a980 con 0x7f273807be10 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f273cd53700 1 --2- 172.21.15.145:0/2522146475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f272c0098a0 tx=0x7f272c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd947d63093a2e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.680+0000 7f27377fe700 1 -- 172.21.15.145:0/2522146475 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f272c004160 con 0x7f273807be10 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f27377fe700 1 -- 172.21.15.145:0/2522146475 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f272c0042c0 con 0x7f273807be10 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f27377fe700 1 -- 172.21.15.145:0/2522146475 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f272c011a00 con 0x7f273807be10 2023-08-25T04:36:19.337 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 -- 172.21.15.145:0/2522146475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 msgr2=0x7f273807a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 --2- 172.21.15.145:0/2522146475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f272c0098a0 tx=0x7f272c009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 -- 172.21.15.145:0/2522146475 shutdown_connections 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 --2- 172.21.15.145:0/2522146475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273807a270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 -- 172.21.15.145:0/2522146475 >> 172.21.15.145:0/2522146475 conn(0x7f2738101750 msgr2=0x7f2738103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 -- 172.21.15.145:0/2522146475 shutdown_connections 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.681+0000 7f273efb7700 1 -- 172.21.15.145:0/2522146475 wait complete. 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.682+0000 7f273efb7700 1 Processor -- start 2023-08-25T04:36:19.338 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.682+0000 7f273efb7700 1 -- start start 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273efb7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273efb7700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273819c5c0 con 0x7f273807be10 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273cd53700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273cd53700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60716/0 (socket says 172.21.15.145:60716) 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273cd53700 1 -- 172.21.15.145:0/2779851874 learned_addr learned my addr 172.21.15.145:0/2779851874 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273cd53700 1 -- 172.21.15.145:0/2779851874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c009550 con 0x7f273807be10 2023-08-25T04:36:19.339 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.683+0000 7f273cd53700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f272c006af0 tx=0x7f272c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.684+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f272c003a40 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.684+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f272c003ba0 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.684+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f272c01a6e0 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.684+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273819c7c0 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.684+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273819cbe0 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.685+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f272c024a60 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.685+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2738061f50 con 0x7f273807be10 2023-08-25T04:36:19.340 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.685+0000 7f2735ffb700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 0x7f272803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.685+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f272c04bff0 con 0x7f273807be10 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.686+0000 7f2737fff700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 0x7f272803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.686+0000 7f2737fff700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 0x7f272803a620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2724006fd0 tx=0x7f2724006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.691+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f272c024430 con 0x7f273807be10 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.919+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2738103a50 con 0x7f2728038160 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.925+0000 7f2735ffb700 1 -- 172.21.15.145:0/2779851874 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2738103a50 con 0x7f2728038160 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 msgr2=0x7f272803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:19.341 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 0x7f272803a620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2724006fd0 tx=0x7f2724006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 msgr2=0x7f273819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f272c006af0 tx=0x7f272c011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 shutdown_connections 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2728038160 0x7f272803a620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 --2- 172.21.15.145:0/2779851874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f273807be10 0x7f273819c080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 >> 172.21.15.145:0/2779851874 conn(0x7f2738101750 msgr2=0x7f2738102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:19.342 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 shutdown_connections 2023-08-25T04:36:19.343 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:18.928+0000 7f273efb7700 1 -- 172.21.15.145:0/2779851874 wait complete. 2023-08-25T04:36:19.343 INFO:teuthology.orchestra.run.smithi145.stdout:Deploying unmanaged mgr service... 2023-08-25T04:36:19.955 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:19 smithi145 ceph-mon[97681]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:19.956 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:19 smithi145 ceph-mon[97681]: Saving service mon spec with placement count:5 2023-08-25T04:36:19.956 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:19 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:20.722 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-08-25T04:36:20.722 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.945+0000 7fb5e728a700 1 Processor -- start 2023-08-25T04:36:20.722 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.945+0000 7fb5e728a700 1 -- start start 2023-08-25T04:36:20.722 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e728a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:20.722 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e728a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e007a840 con 0x7fb5e007be10 2023-08-25T04:36:20.723 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e5026700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:20.723 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e5026700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60726/0 (socket says 172.21.15.145:60726) 2023-08-25T04:36:20.723 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e5026700 1 -- 172.21.15.145:0/3031550120 learned_addr learned my addr 172.21.15.145:0/3031550120 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.946+0000 7fb5e5026700 1 -- 172.21.15.145:0/3031550120 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e007a980 con 0x7fb5e007be10 2023-08-25T04:36:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5e5026700 1 --2- 172.21.15.145:0/3031550120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0098a0 tx=0x7fb5dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a16ccf0984a47 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5d7fff700 1 -- 172.21.15.145:0/3031550120 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5dc004160 con 0x7fb5e007be10 2023-08-25T04:36:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5d7fff700 1 -- 172.21.15.145:0/3031550120 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5dc0042c0 con 0x7fb5e007be10 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5d7fff700 1 -- 172.21.15.145:0/3031550120 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5dc011a00 con 0x7fb5e007be10 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5e728a700 1 -- 172.21.15.145:0/3031550120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 msgr2=0x7fb5e007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.947+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3031550120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0098a0 tx=0x7fb5dc009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.948+0000 7fb5e728a700 1 -- 172.21.15.145:0/3031550120 shutdown_connections 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.948+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3031550120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e007a270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.948+0000 7fb5e728a700 1 -- 172.21.15.145:0/3031550120 >> 172.21.15.145:0/3031550120 conn(0x7fb5e0101750 msgr2=0x7fb5e0103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.948+0000 7fb5e728a700 1 -- 172.21.15.145:0/3031550120 shutdown_connections 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.948+0000 7fb5e728a700 1 -- 172.21.15.145:0/3031550120 wait complete. 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e728a700 1 Processor -- start 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e728a700 1 -- start start 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e728a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:20.726 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e728a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e019c5c0 con 0x7fb5e007be10 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e5026700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e5026700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60742/0 (socket says 172.21.15.145:60742) 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.949+0000 7fb5e5026700 1 -- 172.21.15.145:0/3540106348 learned_addr learned my addr 172.21.15.145:0/3540106348 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5e5026700 1 -- 172.21.15.145:0/3540106348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc009550 con 0x7fb5e007be10 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5e5026700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006af0 tx=0x7fb5dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5dc003a40 con 0x7fb5e007be10 2023-08-25T04:36:20.727 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5dc003ba0 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e019c7c0 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5dc01a6e0 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.950+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e019cbe0 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.951+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fb5dc024430 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.951+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e0061f50 con 0x7fb5e007be10 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.952+0000 7fb5d67fc700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 0x7fb5cc03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:20.728 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.952+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb5dc04bfa0 con 0x7fb5e007be10 2023-08-25T04:36:20.729 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.952+0000 7fb5e4825700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 0x7fb5cc03a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:20.729 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.952+0000 7fb5e4825700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 0x7fb5cc03a620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0006fd0 tx=0x7fb5d0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:20.729 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:19.957+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb5dc0246e0 con 0x7fb5e007be10 2023-08-25T04:36:20.729 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.174+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb5e0103a50 con 0x7fb5cc038160 2023-08-25T04:36:20.729 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.181+0000 7fb5d67fc700 1 -- 172.21.15.145:0/3540106348 <== mgr.14118 v2:172.21.15.145:6800/3601102351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb5e0103a50 con 0x7fb5cc038160 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 msgr2=0x7fb5cc03a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 0x7fb5cc03a620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0006fd0 tx=0x7fb5d0006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 msgr2=0x7fb5e019c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006af0 tx=0x7fb5dc011ad0 comp rx=0 tx=0).stop 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 shutdown_connections 2023-08-25T04:36:20.730 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fb5cc038160 0x7fb5cc03a620 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:20.731 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 --2- 172.21.15.145:0/3540106348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e007be10 0x7fb5e019c080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:20.731 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 >> 172.21.15.145:0/3540106348 conn(0x7fb5e0101750 msgr2=0x7fb5e0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:20.731 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 shutdown_connections 2023-08-25T04:36:20.731 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:20.184+0000 7fb5e728a700 1 -- 172.21.15.145:0/3540106348 wait complete. 2023-08-25T04:36:21.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:21 smithi145 ceph-mon[97681]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:21.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:21 smithi145 ceph-mon[97681]: Saving service mgr spec with placement count:2 2023-08-25T04:36:21.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:21 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:21.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:21 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:21.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:21 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:22.101 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.277+0000 7f1d46d8e700 1 Processor -- start 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d46d8e700 1 -- start start 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d46d8e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d46d8e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4007a840 con 0x7f1d4007be10 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d44b2a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d44b2a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60746/0 (socket says 172.21.15.145:60746) 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.278+0000 7f1d44b2a700 1 -- 172.21.15.145:0/411868787 learned_addr learned my addr 172.21.15.145:0/411868787 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:22.102 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.279+0000 7f1d44b2a700 1 -- 172.21.15.145:0/411868787 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4007a980 con 0x7f1d4007be10 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.279+0000 7f1d44b2a700 1 --2- 172.21.15.145:0/411868787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d340098a0 tx=0x7f1d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fa3f3288aeccf2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.279+0000 7f1d3f7fe700 1 -- 172.21.15.145:0/411868787 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d34004160 con 0x7f1d4007be10 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.279+0000 7f1d3f7fe700 1 -- 172.21.15.145:0/411868787 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d340042c0 con 0x7f1d4007be10 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.279+0000 7f1d3f7fe700 1 -- 172.21.15.145:0/411868787 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d340119b0 con 0x7f1d4007be10 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 -- 172.21.15.145:0/411868787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 msgr2=0x7f1d4007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/411868787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d340098a0 tx=0x7f1d34009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:22.103 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 -- 172.21.15.145:0/411868787 shutdown_connections 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/411868787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4007a270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 -- 172.21.15.145:0/411868787 >> 172.21.15.145:0/411868787 conn(0x7f1d40101750 msgr2=0x7f1d40103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 -- 172.21.15.145:0/411868787 shutdown_connections 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.280+0000 7f1d46d8e700 1 -- 172.21.15.145:0/411868787 wait complete. 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.281+0000 7f1d46d8e700 1 Processor -- start 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.281+0000 7f1d46d8e700 1 -- start start 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d46d8e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:22.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d46d8e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4019c5a0 con 0x7f1d4007be10 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d44b2a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d44b2a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60762/0 (socket says 172.21.15.145:60762) 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d44b2a700 1 -- 172.21.15.145:0/4277791445 learned_addr learned my addr 172.21.15.145:0/4277791445 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d44b2a700 1 -- 172.21.15.145:0/4277791445 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009550 con 0x7f1d4007be10 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.282+0000 7f1d44b2a700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d34003e60 tx=0x7f1d34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.283+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d34003970 con 0x7f1d4007be10 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.283+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d34003ad0 con 0x7f1d4007be10 2023-08-25T04:36:22.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.283+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4019c7a0 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.283+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d340195e0 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.283+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4019cbc0 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.284+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1d3401a430 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.284+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40061f50 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.284+0000 7f1d3dffb700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 0x7f1d3003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.284+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1d3404c120 con 0x7f1d4007be10 2023-08-25T04:36:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.285+0000 7f1d3ffff700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 0x7f1d3003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.285+0000 7f1d3ffff700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 0x7f1d3003a5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c006fd0 tx=0x7f1d2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.289+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1d34003c40 con 0x7f1d4007be10 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.492+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f1d4010c5f0 con 0x7f1d4007be10 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.498+0000 7f1d3dffb700 1 -- 172.21.15.145:0/4277791445 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1d34023b80 con 0x7f1d4007be10 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 msgr2=0x7f1d3003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 0x7f1d3003a5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c006fd0 tx=0x7f1d2c006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 msgr2=0x7f1d4019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d34003e60 tx=0x7f1d34011aa0 comp rx=0 tx=0).stop 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 shutdown_connections 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1d30038110 0x7f1d3003a5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 --2- 172.21.15.145:0/4277791445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d4007be10 0x7f1d4019c060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.501+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 >> 172.21.15.145:0/4277791445 conn(0x7f1d40101750 msgr2=0x7f1d401024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.502+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 shutdown_connections 2023-08-25T04:36:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:21.502+0000 7f1d46d8e700 1 -- 172.21.15.145:0/4277791445 wait complete. 2023-08-25T04:36:22.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:22 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4277791445' entity='client.admin' 2023-08-25T04:36:23.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.768+0000 7f1e46729700 1 Processor -- start 2023-08-25T04:36:23.486 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.769+0000 7f1e46729700 1 -- start start 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.769+0000 7f1e46729700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.769+0000 7f1e46729700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4007a880 con 0x7f1e4007be50 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60772/0 (socket says 172.21.15.145:60772) 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3ffff700 1 -- 172.21.15.145:0/1193010769 learned_addr learned my addr 172.21.15.145:0/1193010769 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3ffff700 1 -- 172.21.15.145:0/1193010769 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4007a9c0 con 0x7f1e4007be50 2023-08-25T04:36:23.487 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3ffff700 1 --2- 172.21.15.145:0/1193010769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098a0 tx=0x7f1e30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180e85a4e2966b06 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3effd700 1 -- 172.21.15.145:0/1193010769 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e30004160 con 0x7f1e4007be50 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3effd700 1 -- 172.21.15.145:0/1193010769 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e300042c0 con 0x7f1e4007be50 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.770+0000 7f1e3effd700 1 -- 172.21.15.145:0/1193010769 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e300119b0 con 0x7f1e4007be50 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 -- 172.21.15.145:0/1193010769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 msgr2=0x7f1e4007a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 --2- 172.21.15.145:0/1193010769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098a0 tx=0x7f1e30009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 -- 172.21.15.145:0/1193010769 shutdown_connections 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 --2- 172.21.15.145:0/1193010769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4007a2b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:23.488 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 -- 172.21.15.145:0/1193010769 >> 172.21.15.145:0/1193010769 conn(0x7f1e40101750 msgr2=0x7f1e40103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 -- 172.21.15.145:0/1193010769 shutdown_connections 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.771+0000 7f1e46729700 1 -- 172.21.15.145:0/1193010769 wait complete. 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.772+0000 7f1e46729700 1 Processor -- start 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.772+0000 7f1e46729700 1 -- start start 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e46729700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e46729700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4019c5c0 con 0x7f1e4007be50 2023-08-25T04:36:23.489 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60782/0 (socket says 172.21.15.145:60782) 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e3ffff700 1 -- 172.21.15.145:0/924050081 learned_addr learned my addr 172.21.15.145:0/924050081 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e3ffff700 1 -- 172.21.15.145:0/924050081 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009550 con 0x7f1e4007be50 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.773+0000 7f1e3ffff700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e30000c00 tx=0x7f1e30011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.774+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e300039f0 con 0x7f1e4007be50 2023-08-25T04:36:23.490 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.774+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e30003b50 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.774+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4019c7c0 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.774+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e3001a780 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.774+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4019cbe0 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.775+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1e30024440 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.775+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e40061f50 con 0x7f1e4007be50 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.775+0000 7f1e3d7fa700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 0x7f1e2803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:23.491 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.775+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e3004c080 con 0x7f1e4007be50 2023-08-25T04:36:23.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.776+0000 7f1e3f7fe700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 0x7f1e2803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:23.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.776+0000 7f1e3f7fe700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 0x7f1e2803a620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:23.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.780+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1e300246f0 con 0x7f1e4007be50 2023-08-25T04:36:23.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.986+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1e4019f620 con 0x7f1e4007be50 2023-08-25T04:36:23.492 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.992+0000 7f1e3d7fa700 1 -- 172.21.15.145:0/924050081 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1e30018350 con 0x7f1e4007be50 2023-08-25T04:36:23.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 msgr2=0x7f1e2803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:23.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 0x7f1e2803a620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:23.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 msgr2=0x7f1e4019c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:23.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e30000c00 tx=0x7f1e30011a30 comp rx=0 tx=0).stop 2023-08-25T04:36:23.493 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 shutdown_connections 2023-08-25T04:36:23.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f1e28038160 0x7f1e2803a620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:23.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 --2- 172.21.15.145:0/924050081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e4007be50 0x7f1e4019c080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:23.494 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.995+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 >> 172.21.15.145:0/924050081 conn(0x7f1e40101750 msgr2=0x7f1e40102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:23.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.996+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 shutdown_connections 2023-08-25T04:36:23.495 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:22.996+0000 7f1e46729700 1 -- 172.21.15.145:0/924050081 wait complete. 2023-08-25T04:36:23.495 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling the dashboard module... 2023-08-25T04:36:24.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:23 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:23 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/924050081' entity='client.admin' 2023-08-25T04:36:25.237 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:24 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1623394360' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-08-25T04:36:25.238 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:24 smithi145 ceph-mon[97681]: from='mgr.14118 172.21.15.145:0/156892297' entity='mgr.x' 2023-08-25T04:36:25.238 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setuser ceph since I am not root 2023-08-25T04:36:25.239 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setgroup ceph since I am not root 2023-08-25T04:36:25.472 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.998+0000 7f257085a700 1 Processor -- start 2023-08-25T04:36:25.472 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.998+0000 7f257085a700 1 -- start start 2023-08-25T04:36:25.472 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.998+0000 7f257085a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:25.472 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.998+0000 7f257085a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c106ba0 con 0x7f256c1061b0 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45964/0 (socket says 172.21.15.145:45964) 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256a59c700 1 -- 172.21.15.145:0/3022158843 learned_addr learned my addr 172.21.15.145:0/3022158843 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256a59c700 1 -- 172.21.15.145:0/3022158843 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c1073b0 con 0x7f256c1061b0 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256a59c700 1 --2- 172.21.15.145:0/3022158843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098a0 tx=0x7f255c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3227dde4e61309f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:23.999+0000 7f256959a700 1 -- 172.21.15.145:0/3022158843 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255c004160 con 0x7f256c1061b0 2023-08-25T04:36:25.473 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f256959a700 1 -- 172.21.15.145:0/3022158843 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c0042c0 con 0x7f256c1061b0 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f256959a700 1 -- 172.21.15.145:0/3022158843 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255c0119b0 con 0x7f256c1061b0 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 -- 172.21.15.145:0/3022158843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 msgr2=0x7f256c1065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 --2- 172.21.15.145:0/3022158843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098a0 tx=0x7f255c009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 -- 172.21.15.145:0/3022158843 shutdown_connections 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 --2- 172.21.15.145:0/3022158843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c1065d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 -- 172.21.15.145:0/3022158843 >> 172.21.15.145:0/3022158843 conn(0x7f256c101750 msgr2=0x7f256c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:25.474 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 -- 172.21.15.145:0/3022158843 shutdown_connections 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.000+0000 7f257085a700 1 -- 172.21.15.145:0/3022158843 wait complete. 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.001+0000 7f257085a700 1 Processor -- start 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f257085a700 1 -- start start 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f257085a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f257085a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c07c120 con 0x7f256c1061b0 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f256a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f256a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45978/0 (socket says 172.21.15.145:45978) 2023-08-25T04:36:25.475 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.002+0000 7f256a59c700 1 -- 172.21.15.145:0/1623394360 learned_addr learned my addr 172.21.15.145:0/1623394360 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f256a59c700 1 -- 172.21.15.145:0/1623394360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009550 con 0x7f256c1061b0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f256a59c700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255c0039f0 con 0x7f256c1061b0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c003b50 con 0x7f256c1061b0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c07a230 con 0x7f256c1061b0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255c01a780 con 0x7f256c1061b0 2023-08-25T04:36:25.476 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.003+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c07a650 con 0x7f256c1061b0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.004+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c061f50 con 0x7f256c1061b0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.005+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f255c024440 con 0x7f256c1061b0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.005+0000 7f255b7fe700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 0x7f255403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.005+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f255c04c040 con 0x7f256c1061b0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.005+0000 7f2569d9b700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 0x7f255403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.006+0000 7f2569d9b700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 0x7f255403a5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.010+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f255c018b80 con 0x7f256c1061b0 2023-08-25T04:36:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.280+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f256c10a630 con 0x7f256c1061b0 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.927+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f255c02d910 con 0x7f256c1061b0 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.930+0000 7f255b7fe700 1 -- 172.21.15.145:0/1623394360 <== mon.0 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f255c02d3f0 con 0x7f256c1061b0 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 msgr2=0x7f255403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 0x7f255403a5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 msgr2=0x7f256c07bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c011a30 comp rx=0 tx=0).stop 2023-08-25T04:36:25.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 shutdown_connections 2023-08-25T04:36:25.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f2554038110 0x7f255403a5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:25.479 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 --2- 172.21.15.145:0/1623394360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c1061b0 0x7f256c07bbe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:25.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.933+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 >> 172.21.15.145:0/1623394360 conn(0x7f256c101750 msgr2=0x7f256c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:25.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.934+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 shutdown_connections 2023-08-25T04:36:25.480 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:24.934+0000 7f257085a700 1 -- 172.21.15.145:0/1623394360 wait complete. 2023-08-25T04:36:25.580 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:25.235+0000 7f1c0c07e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:36:25.581 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:25.341+0000 7f1c0c07e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:36:25.581 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:25.457+0000 7f1c0c07e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:36:25.892 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:25.615+0000 7f1c0c07e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:36:25.892 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:25.853+0000 7f1c0c07e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:36:26.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1623394360' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-08-25T04:36:26.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:25 smithi145 ceph-mon[97681]: mgrmap e9: x(active, since 17s) 2023-08-25T04:36:26.638 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:26.343+0000 7f1c0c07e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:36:26.639 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:26.518+0000 7f1c0c07e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:36:26.928 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:26.637+0000 7f1c0c07e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:36:26.929 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:26.766+0000 7f1c0c07e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:36:27.152 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:36:27.152 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-08-25T04:36:27.152 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-25T04:36:27.152 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-25T04:36:27.153 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-25T04:36:27.153 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:36:27.153 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.412+0000 7f316705b700 1 Processor -- start 2023-08-25T04:36:27.153 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.412+0000 7f316705b700 1 -- start start 2023-08-25T04:36:27.153 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.413+0000 7f316705b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.413+0000 7f316705b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316007a840 con 0x7f316007be10 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.413+0000 7f3164df7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.413+0000 7f3164df7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45996/0 (socket says 172.21.15.145:45996) 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.413+0000 7f3164df7700 1 -- 172.21.15.145:0/3461964279 learned_addr learned my addr 172.21.15.145:0/3461964279 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.414+0000 7f3164df7700 1 -- 172.21.15.145:0/3461964279 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316007a980 con 0x7f316007be10 2023-08-25T04:36:27.154 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.414+0000 7f3164df7700 1 --2- 172.21.15.145:0/3461964279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31540098a0 tx=0x7f3154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c07134418e8b23d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:27.155 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.414+0000 7f315f7fe700 1 -- 172.21.15.145:0/3461964279 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3154004160 con 0x7f316007be10 2023-08-25T04:36:27.155 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.414+0000 7f315f7fe700 1 -- 172.21.15.145:0/3461964279 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31540042c0 con 0x7f316007be10 2023-08-25T04:36:27.155 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.414+0000 7f315f7fe700 1 -- 172.21.15.145:0/3461964279 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31540119b0 con 0x7f316007be10 2023-08-25T04:36:27.156 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 -- 172.21.15.145:0/3461964279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 msgr2=0x7f316007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:27.156 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 --2- 172.21.15.145:0/3461964279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31540098a0 tx=0x7f3154009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:27.156 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 -- 172.21.15.145:0/3461964279 shutdown_connections 2023-08-25T04:36:27.156 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 --2- 172.21.15.145:0/3461964279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316007a270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 -- 172.21.15.145:0/3461964279 >> 172.21.15.145:0/3461964279 conn(0x7f3160101750 msgr2=0x7f3160103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 -- 172.21.15.145:0/3461964279 shutdown_connections 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.415+0000 7f316705b700 1 -- 172.21.15.145:0/3461964279 wait complete. 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.416+0000 7f316705b700 1 Processor -- start 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.416+0000 7f316705b700 1 -- start start 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.416+0000 7f316705b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:27.157 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.416+0000 7f316705b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019c5a0 con 0x7f316007be10 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f3164df7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f3164df7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46008/0 (socket says 172.21.15.145:46008) 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f3164df7700 1 -- 172.21.15.145:0/1944412618 learned_addr learned my addr 172.21.15.145:0/1944412618 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f3164df7700 1 -- 172.21.15.145:0/1944412618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154009550 con 0x7f316007be10 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f3164df7700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3154011980 tx=0x7f3154011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.417+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3154003a60 con 0x7f316007be10 2023-08-25T04:36:27.158 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.418+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316019c7a0 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.418+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154003bc0 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.418+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f315401a670 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.418+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316019cbc0 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.419+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f3154024440 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.419+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3160061f50 con 0x7f316007be10 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.419+0000 7f315dffb700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 0x7f315003a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:27.159 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.419+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f315404c0c0 con 0x7f316007be10 2023-08-25T04:36:27.160 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.420+0000 7f315ffff700 1 -- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 msgr2=0x7f315003a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:27.160 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.420+0000 7f315ffff700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 0x7f315003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:36:27.160 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.425+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f31540189a0 con 0x7f316007be10 2023-08-25T04:36:27.160 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.621+0000 7f315ffff700 1 -- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 msgr2=0x7f315003a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:27.161 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.621+0000 7f315ffff700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 0x7f315003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:36:27.161 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.688+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f316010c5f0 con 0x7f316007be10 2023-08-25T04:36:27.161 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.689+0000 7f315dffb700 1 -- 172.21.15.145:0/1944412618 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3154018350 con 0x7f316007be10 2023-08-25T04:36:27.161 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.691+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 msgr2=0x7f315003a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:36:27.162 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.691+0000 7f316705b700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 0x7f315003a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:27.162 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.691+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 msgr2=0x7f316019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:27.162 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.691+0000 7f316705b700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3154011980 tx=0x7f3154011a60 comp rx=0 tx=0).stop 2023-08-25T04:36:27.163 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 shutdown_connections 2023-08-25T04:36:27.163 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7f3150038160 0x7f315003a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:27.163 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 --2- 172.21.15.145:0/1944412618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f316007be10 0x7f316019c060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:27.164 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 >> 172.21.15.145:0/1944412618 conn(0x7f3160101750 msgr2=0x7f3160102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:27.164 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 shutdown_connections 2023-08-25T04:36:27.164 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:26.692+0000 7f316705b700 1 -- 172.21.15.145:0/1944412618 wait complete. 2023-08-25T04:36:27.165 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for the mgr to restart... 2023-08-25T04:36:27.165 INFO:teuthology.orchestra.run.smithi145.stdout:Waiting for mgr epoch 9... 2023-08-25T04:36:27.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1944412618' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-25T04:36:27.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:27.351+0000 7f1c0c07e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:36:28.577 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:28.220+0000 7f1c0c07e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:36:28.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:28.325+0000 7f1c0c07e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:36:29.577 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:29.274+0000 7f1c0c07e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:36:29.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:29.391+0000 7f1c0c07e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:36:29.977 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:29.614+0000 7f1c0c07e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:36:30.256 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:29.976+0000 7f1c0c07e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:36:30.577 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:30.255+0000 7f1c0c07e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:36:30.578 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:30.447+0000 7f1c0c07e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:36:31.077 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:30.790+0000 7f1c0c07e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:36:32.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:31.710+0000 7f1c0c07e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:36:32.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:31.881+0000 7f1c0c07e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:36:32.545 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:32.128+0000 7f1c0c07e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:36:32.545 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:32.271+0000 7f1c0c07e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:36:32.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:32.544+0000 7f1c0c07e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:36:34.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:36:33.604+0000 7f1c0c07e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:36:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: Active manager daemon x restarted 2023-08-25T04:36:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: Activating manager daemon x 2023-08-25T04:36:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: osdmap e3: 0 total, 0 up, 0 in 2023-08-25T04:36:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: mgrmap e10: x(active, starting, since 0.00413063s) 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: Manager daemon x is now available 2023-08-25T04:36:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:34.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:35.047 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:36:35.047 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:36:35.047 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:33] ENGINE Bus STARTING 2023-08-25T04:36:35.047 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:33] ENGINE Serving on https://172.21.15.145:7150 2023-08-25T04:36:35.047 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:34] ENGINE Serving on http://172.21.15.145:8765 2023-08-25T04:36:35.048 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: [25/Aug/2023:04:36:34] ENGINE Bus STARTED 2023-08-25T04:36:35.048 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:34 smithi145 ceph-mon[97681]: mgrmap e11: x(active, since 1.0073s) 2023-08-25T04:36:35.058 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout { 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout } 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.710+0000 7fbc25dc9700 1 Processor -- start 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.710+0000 7fbc25dc9700 1 -- start start 2023-08-25T04:36:35.059 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc25dc9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc25dc9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2007a840 con 0x7fbc2007be10 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc1f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc1f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46016/0 (socket says 172.21.15.145:46016) 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc1f7fe700 1 -- 172.21.15.145:0/421669680 learned_addr learned my addr 172.21.15.145:0/421669680 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.711+0000 7fbc1f7fe700 1 -- 172.21.15.145:0/421669680 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2007a980 con 0x7fbc2007be10 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.712+0000 7fbc1f7fe700 1 --2- 172.21.15.145:0/421669680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098a0 tx=0x7fbc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ff5ca0143a6241 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:35.060 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.712+0000 7fbc1e7fc700 1 -- 172.21.15.145:0/421669680 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc10004160 con 0x7fbc2007be10 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.712+0000 7fbc1e7fc700 1 -- 172.21.15.145:0/421669680 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc100042c0 con 0x7fbc2007be10 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.712+0000 7fbc1e7fc700 1 -- 172.21.15.145:0/421669680 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc100119b0 con 0x7fbc2007be10 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 -- 172.21.15.145:0/421669680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 msgr2=0x7fbc2007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/421669680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098a0 tx=0x7fbc10009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 -- 172.21.15.145:0/421669680 shutdown_connections 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/421669680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2007a270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.061 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 -- 172.21.15.145:0/421669680 >> 172.21.15.145:0/421669680 conn(0x7fbc20101750 msgr2=0x7fbc20103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 -- 172.21.15.145:0/421669680 shutdown_connections 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.713+0000 7fbc25dc9700 1 -- 172.21.15.145:0/421669680 wait complete. 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.714+0000 7fbc25dc9700 1 Processor -- start 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.714+0000 7fbc25dc9700 1 -- start start 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.714+0000 7fbc25dc9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.714+0000 7fbc25dc9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2019c5a0 con 0x7fbc2007be10 2023-08-25T04:36:35.062 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46018/0 (socket says 172.21.15.145:46018) 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1f7fe700 1 -- 172.21.15.145:0/3566264338 learned_addr learned my addr 172.21.15.145:0/3566264338 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1f7fe700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc10009550 con 0x7fbc2007be10 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1f7fe700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc10003e60 tx=0x7fbc10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.715+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc10003970 con 0x7fbc2007be10 2023-08-25T04:36:35.063 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.716+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc10003ad0 con 0x7fbc2007be10 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.716+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc100195e0 con 0x7fbc2007be10 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.716+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2019c7a0 con 0x7fbc2007be10 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.716+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2019cbc0 con 0x7fbc2007be10 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fbc10019850 con 0x7fbc2007be10 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1cff9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc0803ad30 con 0x7fbc08038160 2023-08-25T04:36:35.064 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbc1004c200 con 0x7fbc2007be10 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1effd700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.717+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.918+0000 7fbc1effd700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:27.918+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:28.319+0000 7fbc1effd700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:28.319+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T04:36:35.065 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:29.120+0000 7fbc1effd700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:29.120+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:30.721+0000 7fbc1effd700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/3601102351 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:30.721+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:33.608+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mgrmap(e 10) v1 ==== 44915+0+0 (secure 0 0 0) 0x7fbc10019b50 con 0x7fbc2007be10 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:33.609+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 msgr2=0x7fbc0803a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:33.609+0000 7fbc1cff9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.066 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.610+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 11) v1 ==== 45042+0+0 (secure 0 0 0) 0x7fbc1004db20 con 0x7fbc2007be10 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.610+0000 7fbc1cff9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 0x7fbc0803e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.610+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc0803ad30 con 0x7fbc0803bc10 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.611+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 0x7fbc0803e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.612+0000 7fbc1effd700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 0x7fbc0803e000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbc14003a10 tx=0x7fbc14009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.613+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbc0803ad30 con 0x7fbc0803bc10 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.619+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbc2019d3a0 con 0x7fbc0803bc10 2023-08-25T04:36:35.067 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc1cff9700 1 -- 172.21.15.145:0/3566264338 <== mgr.14150 v2:172.21.15.145:6800/891678096 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fbc2019d3a0 con 0x7fbc0803bc10 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 msgr2=0x7fbc0803e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 0x7fbc0803e000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbc14003a10 tx=0x7fbc14009270 comp rx=0 tx=0).stop 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 msgr2=0x7fbc2019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc10003e60 tx=0x7fbc10011aa0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 shutdown_connections 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2007be10 0x7fbc2019c060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.068 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc0803bc10 0x7fbc0803e000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 --2- 172.21.15.145:0/3566264338 >> [v2:172.21.15.145:6800/3601102351,v1:172.21.15.145:6801/3601102351] conn(0x7fbc08038160 0x7fbc0803a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 >> 172.21.15.145:0/3566264338 conn(0x7fbc20101750 msgr2=0x7fbc20102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 shutdown_connections 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:34.620+0000 7fbc25dc9700 1 -- 172.21.15.145:0/3566264338 wait complete. 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:mgr epoch 9 is available 2023-08-25T04:36:35.069 INFO:teuthology.orchestra.run.smithi145.stdout:Generating a dashboard self-signed certificate... 2023-08-25T04:36:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:35 smithi145 ceph-mon[97681]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-25T04:36:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:35 smithi145 ceph-mon[97681]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-25T04:36:36.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-08-25T04:36:36.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.872+0000 7f19abb3b700 1 Processor -- start 2023-08-25T04:36:36.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19abb3b700 1 -- start start 2023-08-25T04:36:36.640 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19abb3b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19abb3b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4079dc0 con 0x7f19a4106c90 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19a98d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19a98d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42632/0 (socket says 172.21.15.145:42632) 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.873+0000 7f19a98d7700 1 -- 172.21.15.145:0/4268740926 learned_addr learned my addr 172.21.15.145:0/4268740926 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.874+0000 7f19a98d7700 1 -- 172.21.15.145:0/4268740926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a41095c0 con 0x7f19a4106c90 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.874+0000 7f19a98d7700 1 --2- 172.21.15.145:0/4268740926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19a00098a0 tx=0x7f19a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986d334bf0d3619 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:36.641 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.874+0000 7f19a88d5700 1 -- 172.21.15.145:0/4268740926 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a0004160 con 0x7f19a4106c90 2023-08-25T04:36:36.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.874+0000 7f19a88d5700 1 -- 172.21.15.145:0/4268740926 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a00042c0 con 0x7f19a4106c90 2023-08-25T04:36:36.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.874+0000 7f19a88d5700 1 -- 172.21.15.145:0/4268740926 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00119b0 con 0x7f19a4106c90 2023-08-25T04:36:36.642 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 -- 172.21.15.145:0/4268740926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 msgr2=0x7f19a4109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 --2- 172.21.15.145:0/4268740926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19a00098a0 tx=0x7f19a0009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 -- 172.21.15.145:0/4268740926 shutdown_connections 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 --2- 172.21.15.145:0/4268740926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a4109080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 -- 172.21.15.145:0/4268740926 >> 172.21.15.145:0/4268740926 conn(0x7f19a4100620 msgr2=0x7f19a4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 -- 172.21.15.145:0/4268740926 shutdown_connections 2023-08-25T04:36:36.643 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.875+0000 7f19abb3b700 1 -- 172.21.15.145:0/4268740926 wait complete. 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.876+0000 7f19abb3b700 1 Processor -- start 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.876+0000 7f19abb3b700 1 -- start start 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19abb3b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19abb3b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a419c580 con 0x7f19a4106c90 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19a98d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:36.644 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19a98d7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42644/0 (socket says 172.21.15.145:42644) 2023-08-25T04:36:36.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19a98d7700 1 -- 172.21.15.145:0/3581292643 learned_addr learned my addr 172.21.15.145:0/3581292643 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:36.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19a98d7700 1 -- 172.21.15.145:0/3581292643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0009550 con 0x7f19a4106c90 2023-08-25T04:36:36.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.877+0000 7f19a98d7700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19a0003e60 tx=0x7f19a0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:36.645 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.878+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a0003970 con 0x7f19a4106c90 2023-08-25T04:36:36.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.878+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a0003ad0 con 0x7f19a4106c90 2023-08-25T04:36:36.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.878+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00195e0 con 0x7f19a4106c90 2023-08-25T04:36:36.646 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.878+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a419c780 con 0x7f19a4106c90 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.878+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a419cba0 con 0x7f19a4106c90 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.879+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f19a0019850 con 0x7f19a4106c90 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.879+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a4061f50 con 0x7f19a4106c90 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.879+0000 7f199affd700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 0x7f199003a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.879+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19a004c220 con 0x7f19a4106c90 2023-08-25T04:36:36.647 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.880+0000 7f19a90d6700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 0x7f199003a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:36.648 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.880+0000 7f19a90d6700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 0x7f199003a620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1994006fd0 tx=0x7f1994006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:36.648 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:35.884+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f19a0003c40 con 0x7f19a4106c90 2023-08-25T04:36:36.648 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.097+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a4101b20 con 0x7f1990038160 2023-08-25T04:36:36.648 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.247+0000 7f199affd700 1 -- 172.21.15.145:0/3581292643 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f19a4101b20 con 0x7f1990038160 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 msgr2=0x7f199003a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 0x7f199003a620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1994006fd0 tx=0x7f1994006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 msgr2=0x7f19a419c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19a0003e60 tx=0x7f19a0011aa0 comp rx=0 tx=0).stop 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 shutdown_connections 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1990038160 0x7f199003a620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 --2- 172.21.15.145:0/3581292643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4106c90 0x7f19a419c040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:36.649 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 >> 172.21.15.145:0/3581292643 conn(0x7f19a4100620 msgr2=0x7f19a4101410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:36.650 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 shutdown_connections 2023-08-25T04:36:36.650 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:36.249+0000 7f19abb3b700 1 -- 172.21.15.145:0/3581292643 wait complete. 2023-08-25T04:36:36.650 INFO:teuthology.orchestra.run.smithi145.stdout:Creating initial admin user... 2023-08-25T04:36:36.934 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:36 smithi145 ceph-mon[97681]: mgrmap e12: x(active, since 2s) 2023-08-25T04:36:36.934 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:36.935 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:36.935 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:37 smithi145 ceph-mon[97681]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:38.128 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iHbh571jch/kVP3SNEK/cuGvCOxUfavJj6pHnHwVpURp6tkp0dy8.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1692938197, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-08-25T04:36:38.128 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.209+0000 7ff7fc157700 1 Processor -- start 2023-08-25T04:36:38.128 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.209+0000 7ff7fc157700 1 -- start start 2023-08-25T04:36:38.128 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.209+0000 7ff7fc157700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:38.129 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.209+0000 7ff7fc157700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f407a820 con 0x7ff7f407bdf0 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.210+0000 7ff7f9ef3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.210+0000 7ff7f9ef3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42656/0 (socket says 172.21.15.145:42656) 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.210+0000 7ff7f9ef3700 1 -- 172.21.15.145:0/1743995500 learned_addr learned my addr 172.21.15.145:0/1743995500 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.210+0000 7ff7f9ef3700 1 -- 172.21.15.145:0/1743995500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f407a960 con 0x7ff7f407bdf0 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.210+0000 7ff7f9ef3700 1 --2- 172.21.15.145:0/1743995500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009e80 tx=0x7ff7f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95217ba23df448fd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7f8ef1700 1 -- 172.21.15.145:0/1743995500 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7f000e070 con 0x7ff7f407bdf0 2023-08-25T04:36:38.130 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7f8ef1700 1 -- 172.21.15.145:0/1743995500 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f000b8c0 con 0x7ff7f407bdf0 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7f8ef1700 1 -- 172.21.15.145:0/1743995500 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7f00043e0 con 0x7ff7f407bdf0 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 -- 172.21.15.145:0/1743995500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 msgr2=0x7ff7f407a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 --2- 172.21.15.145:0/1743995500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009e80 tx=0x7ff7f000b330 comp rx=0 tx=0).stop 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 -- 172.21.15.145:0/1743995500 shutdown_connections 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 --2- 172.21.15.145:0/1743995500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f407a250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 -- 172.21.15.145:0/1743995500 >> 172.21.15.145:0/1743995500 conn(0x7ff7f4101750 msgr2=0x7ff7f4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:38.131 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.211+0000 7ff7fc157700 1 -- 172.21.15.145:0/1743995500 shutdown_connections 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.212+0000 7ff7fc157700 1 -- 172.21.15.145:0/1743995500 wait complete. 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.212+0000 7ff7fc157700 1 Processor -- start 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7fc157700 1 -- start start 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7fc157700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7fc157700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f419c5c0 con 0x7ff7f407bdf0 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7f9ef3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:38.132 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7f9ef3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42662/0 (socket says 172.21.15.145:42662) 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.213+0000 7ff7f9ef3700 1 -- 172.21.15.145:0/2995634075 learned_addr learned my addr 172.21.15.145:0/2995634075 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7f9ef3700 1 -- 172.21.15.145:0/2995634075 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009b30 con 0x7ff7f407bdf0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7f9ef3700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0004000 tx=0x7ff7f0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7f000e040 con 0x7ff7f407bdf0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f000be30 con 0x7ff7f407bdf0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f419c7c0 con 0x7ff7f407bdf0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7f001c700 con 0x7ff7f407bdf0 2023-08-25T04:36:38.133 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.214+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f419cc40 con 0x7ff7f407bdf0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.216+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4061f50 con 0x7ff7f407bdf0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.216+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7ff7f0012430 con 0x7ff7f407bdf0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.216+0000 7ff7eaffd700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 0x7ff7e003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.216+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff7f004bf50 con 0x7ff7f407bdf0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.217+0000 7ff7f96f2700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 0x7ff7e003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.217+0000 7ff7f96f2700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 0x7ff7e003a670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006fd0 tx=0x7ff7e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:38.134 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.222+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff7f0011390 con 0x7ff7f407bdf0 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.448+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff7f4061d70 con 0x7ff7e00381b0 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.677+0000 7ff7eaffd700 1 -- 172.21.15.145:0/2995634075 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff7f4061d70 con 0x7ff7e00381b0 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.679+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 msgr2=0x7ff7e003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.679+0000 7ff7fc157700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 0x7ff7e003a670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006fd0 tx=0x7ff7e4006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.679+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 msgr2=0x7ff7f419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.679+0000 7ff7fc157700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0004000 tx=0x7ff7f0003d00 comp rx=0 tx=0).stop 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 shutdown_connections 2023-08-25T04:36:38.135 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff7e00381b0 0x7ff7e003a670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:38.136 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 --2- 172.21.15.145:0/2995634075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f407bdf0 0x7ff7f419c080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:38.136 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 >> 172.21.15.145:0/2995634075 conn(0x7ff7f4101750 msgr2=0x7ff7f4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:38.136 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 shutdown_connections 2023-08-25T04:36:38.136 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:37.680+0000 7ff7fc157700 1 -- 172.21.15.145:0/2995634075 wait complete. 2023-08-25T04:36:38.136 INFO:teuthology.orchestra.run.smithi145.stdout:Fetching dashboard port number... 2023-08-25T04:36:38.728 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:38 smithi145 ceph-mon[97681]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:38.728 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:39.607 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stdout 8443 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.694+0000 7f32adc00700 1 Processor -- start 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.694+0000 7f32adc00700 1 -- start start 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32adc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32adc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8105a90 con 0x7f32a81050a0 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32a77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:39.608 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32a77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42664/0 (socket says 172.21.15.145:42664) 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32a77fe700 1 -- 172.21.15.145:0/1579115148 learned_addr learned my addr 172.21.15.145:0/1579115148 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.695+0000 7f32a77fe700 1 -- 172.21.15.145:0/1579115148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a81062a0 con 0x7f32a81050a0 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.696+0000 7f32a77fe700 1 --2- 172.21.15.145:0/1579115148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32980098a0 tx=0x7f3298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461a86d3d6aa8224 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.696+0000 7f32a67fc700 1 -- 172.21.15.145:0/1579115148 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3298004160 con 0x7f32a81050a0 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.696+0000 7f32a67fc700 1 -- 172.21.15.145:0/1579115148 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32980042c0 con 0x7f32a81050a0 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.696+0000 7f32a67fc700 1 -- 172.21.15.145:0/1579115148 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32980119b0 con 0x7f32a81050a0 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 -- 172.21.15.145:0/1579115148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 msgr2=0x7f32a81054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:39.609 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 --2- 172.21.15.145:0/1579115148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32980098a0 tx=0x7f3298009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 -- 172.21.15.145:0/1579115148 shutdown_connections 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 --2- 172.21.15.145:0/1579115148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a81054c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 -- 172.21.15.145:0/1579115148 >> 172.21.15.145:0/1579115148 conn(0x7f32a8100620 msgr2=0x7f32a8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 -- 172.21.15.145:0/1579115148 shutdown_connections 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.697+0000 7f32adc00700 1 -- 172.21.15.145:0/1579115148 wait complete. 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.698+0000 7f32adc00700 1 Processor -- start 2023-08-25T04:36:39.610 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.698+0000 7f32adc00700 1 -- start start 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32adc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32adc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a81981d0 con 0x7f32a81050a0 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32a77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32a77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42674/0 (socket says 172.21.15.145:42674) 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32a77fe700 1 -- 172.21.15.145:0/3214627152 learned_addr learned my addr 172.21.15.145:0/3214627152 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32a77fe700 1 -- 172.21.15.145:0/3214627152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298009550 con 0x7f32a81050a0 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.699+0000 7f32a77fe700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3298000c00 tx=0x7f3298011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:39.611 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.700+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32980039f0 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.700+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a81983d0 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.700+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3298003b50 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.700+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329801a780 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.700+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a81987f0 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.701+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a8061f50 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.701+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f329801a9f0 con 0x7f32a81050a0 2023-08-25T04:36:39.612 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.702+0000 7f32a4ff9700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 0x7f329003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.702+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f329804c110 con 0x7f32a81050a0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.702+0000 7f32a6ffd700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 0x7f329003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.703+0000 7f32a6ffd700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 0x7f329003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.710+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f32980237b0 con 0x7f32a81050a0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.938+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f32a819b230 con 0x7f32a81050a0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.939+0000 7f32a4ff9700 1 -- 172.21.15.145:0/3214627152 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f329802c350 con 0x7f32a81050a0 2023-08-25T04:36:39.613 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.941+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 msgr2=0x7f329003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.941+0000 7f32adc00700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 0x7f329003a670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.941+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 msgr2=0x7f32a8197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.941+0000 7f32adc00700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3298000c00 tx=0x7f3298011a30 comp rx=0 tx=0).stop 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 shutdown_connections 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f32900381b0 0x7f329003a670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 --2- 172.21.15.145:0/3214627152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32a81050a0 0x7f32a8197c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 >> 172.21.15.145:0/3214627152 conn(0x7f32a8100620 msgr2=0x7f32a80790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:39.614 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 shutdown_connections 2023-08-25T04:36:39.615 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:38.942+0000 7f32adc00700 1 -- 172.21.15.145:0/3214627152 wait complete. 2023-08-25T04:36:39.626 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-25T04:36:39.626 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout disabled 2023-08-25T04:36:39.642 INFO:teuthology.orchestra.run.smithi145.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-25T04:36:39.642 INFO:teuthology.orchestra.run.smithi145.stdout:systemctl: stdout inactive 2023-08-25T04:36:39.642 INFO:teuthology.orchestra.run.smithi145.stdout:firewalld.service is not enabled 2023-08-25T04:36:39.643 INFO:teuthology.orchestra.run.smithi145.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-08-25T04:36:39.644 INFO:teuthology.orchestra.run.smithi145.stdout:Ceph Dashboard is now available at: 2023-08-25T04:36:39.644 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:39.644 INFO:teuthology.orchestra.run.smithi145.stdout: URL: https://smithi145.front.sepia.ceph.com:8443/ 2023-08-25T04:36:39.644 INFO:teuthology.orchestra.run.smithi145.stdout: User: admin 2023-08-25T04:36:39.644 INFO:teuthology.orchestra.run.smithi145.stdout: Password: ypnjlhj9rt 2023-08-25T04:36:39.645 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:39.645 INFO:teuthology.orchestra.run.smithi145.stdout:Saving cluster configuration to /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config directory 2023-08-25T04:36:39.646 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling autotune for osd_memory_target 2023-08-25T04:36:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:39 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3214627152' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-08-25T04:36:40.826 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.198+0000 7f036b93e700 1 Processor -- start 2023-08-25T04:36:40.826 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.198+0000 7f036b93e700 1 -- start start 2023-08-25T04:36:40.826 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.198+0000 7f036b93e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:40.826 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.198+0000 7f036b93e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364106ba0 con 0x7f03641061b0 2023-08-25T04:36:40.826 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.199+0000 7f03696da700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.199+0000 7f03696da700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42684/0 (socket says 172.21.15.145:42684) 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.199+0000 7f03696da700 1 -- 172.21.15.145:0/308177803 learned_addr learned my addr 172.21.15.145:0/308177803 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.199+0000 7f03696da700 1 -- 172.21.15.145:0/308177803 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03641073b0 con 0x7f03641061b0 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.199+0000 7f03696da700 1 --2- 172.21.15.145:0/308177803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03600098a0 tx=0x7f0360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c61abcfef28e1f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f035bfff700 1 -- 172.21.15.145:0/308177803 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0360004160 con 0x7f03641061b0 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f035bfff700 1 -- 172.21.15.145:0/308177803 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03600042c0 con 0x7f03641061b0 2023-08-25T04:36:40.827 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f035bfff700 1 -- 172.21.15.145:0/308177803 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03600119b0 con 0x7f03641061b0 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 -- 172.21.15.145:0/308177803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 msgr2=0x7f03641065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 --2- 172.21.15.145:0/308177803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03600098a0 tx=0x7f0360009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 -- 172.21.15.145:0/308177803 shutdown_connections 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 --2- 172.21.15.145:0/308177803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f03641065d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 -- 172.21.15.145:0/308177803 >> 172.21.15.145:0/308177803 conn(0x7f0364101750 msgr2=0x7f0364103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 -- 172.21.15.145:0/308177803 shutdown_connections 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.200+0000 7f036b93e700 1 -- 172.21.15.145:0/308177803 wait complete. 2023-08-25T04:36:40.828 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.201+0000 7f036b93e700 1 Processor -- start 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.201+0000 7f036b93e700 1 -- start start 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f036b93e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f036b93e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036419c610 con 0x7f03641061b0 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f03696da700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f03696da700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42698/0 (socket says 172.21.15.145:42698) 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f03696da700 1 -- 172.21.15.145:0/1182614891 learned_addr learned my addr 172.21.15.145:0/1182614891 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:40.829 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f03696da700 1 -- 172.21.15.145:0/1182614891 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009550 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.202+0000 7f03696da700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0360000c00 tx=0x7f0360011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.203+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03600039f0 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.203+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0360003b50 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.203+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f036001a780 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.203+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036419c810 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.203+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036419cc30 con 0x7f03641061b0 2023-08-25T04:36:40.830 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.204+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f0360024440 con 0x7f03641061b0 2023-08-25T04:36:40.832 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.204+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0364061f50 con 0x7f03641061b0 2023-08-25T04:36:40.832 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.205+0000 7f035a7fc700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 0x7f035003a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:40.832 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.205+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f036004c170 con 0x7f03641061b0 2023-08-25T04:36:40.832 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.205+0000 7f0368ed9700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 0x7f035003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.205+0000 7f0368ed9700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 0x7f035003a670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.210+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0360018d90 con 0x7f03641061b0 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.404+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f036419f670 con 0x7f03641061b0 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.404+0000 7f035a7fc700 1 -- 172.21.15.145:0/1182614891 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0360018350 con 0x7f03641061b0 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 msgr2=0x7f035003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 0x7f035003a670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 msgr2=0x7f036419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:40.833 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0360000c00 tx=0x7f0360011a30 comp rx=0 tx=0).stop 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 shutdown_connections 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f03500381b0 0x7f035003a670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 --2- 172.21.15.145:0/1182614891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03641061b0 0x7f036419c0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 >> 172.21.15.145:0/1182614891 conn(0x7f0364101750 msgr2=0x7f0364102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 shutdown_connections 2023-08-25T04:36:40.834 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:40.407+0000 7f036b93e700 1 -- 172.21.15.145:0/1182614891 wait complete. 2023-08-25T04:36:41.724 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:41 smithi145 ceph-mon[97681]: mgrmap e13: x(active, since 6s) 2023-08-25T04:36:42.104 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99f0224700 1 Processor -- start 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99f0224700 1 -- start start 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99f0224700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99f0224700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e807a860 con 0x7f99e807be30 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99edfc0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99edfc0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42700/0 (socket says 172.21.15.145:42700) 2023-08-25T04:36:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.372+0000 7f99edfc0700 1 -- 172.21.15.145:0/3606612699 learned_addr learned my addr 172.21.15.145:0/3606612699 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.373+0000 7f99edfc0700 1 -- 172.21.15.145:0/3606612699 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e807a9a0 con 0x7f99e807be30 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.373+0000 7f99edfc0700 1 --2- 172.21.15.145:0/3606612699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99e40098a0 tx=0x7f99e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc0cbe2ce108726 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.373+0000 7f99ecfbe700 1 -- 172.21.15.145:0/3606612699 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e4004160 con 0x7f99e807be30 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.373+0000 7f99ecfbe700 1 -- 172.21.15.145:0/3606612699 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e40042c0 con 0x7f99e807be30 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.373+0000 7f99ecfbe700 1 -- 172.21.15.145:0/3606612699 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e40119b0 con 0x7f99e807be30 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 -- 172.21.15.145:0/3606612699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 msgr2=0x7f99e807a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:42.106 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 --2- 172.21.15.145:0/3606612699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99e40098a0 tx=0x7f99e4009bb0 comp rx=0 tx=0).stop 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 -- 172.21.15.145:0/3606612699 shutdown_connections 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 --2- 172.21.15.145:0/3606612699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e807a290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 -- 172.21.15.145:0/3606612699 >> 172.21.15.145:0/3606612699 conn(0x7f99e8101750 msgr2=0x7f99e8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 -- 172.21.15.145:0/3606612699 shutdown_connections 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.374+0000 7f99f0224700 1 -- 172.21.15.145:0/3606612699 wait complete. 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.375+0000 7f99f0224700 1 Processor -- start 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.375+0000 7f99f0224700 1 -- start start 2023-08-25T04:36:42.107 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.375+0000 7f99f0224700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.375+0000 7f99f0224700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e819c5f0 con 0x7f99e807be30 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99edfc0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99edfc0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42704/0 (socket says 172.21.15.145:42704) 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99edfc0700 1 -- 172.21.15.145:0/2419169456 learned_addr learned my addr 172.21.15.145:0/2419169456 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99edfc0700 1 -- 172.21.15.145:0/2419169456 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4009550 con 0x7f99e807be30 2023-08-25T04:36:42.108 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99edfc0700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e40039f0 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e4003b50 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e819c7f0 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.376+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e401a780 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.377+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e819cc10 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.378+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f99e4024440 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.378+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e810c5d0 con 0x7f99e807be30 2023-08-25T04:36:42.109 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.378+0000 7f99deffd700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 0x7f99d403a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:42.110 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.378+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99e404c220 con 0x7f99e807be30 2023-08-25T04:36:42.110 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.378+0000 7f99ed7bf700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 0x7f99d403a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:42.110 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.379+0000 7f99ed7bf700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 0x7f99d403a710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99d8006fd0 tx=0x7f99d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:42.110 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.383+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f99e40189a0 con 0x7f99e807be30 2023-08-25T04:36:42.110 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.694+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f99e810c810 con 0x7f99e807be30 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.699+0000 7f99deffd700 1 -- 172.21.15.145:0/2419169456 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f99e4018350 con 0x7f99e807be30 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 msgr2=0x7f99d403a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 0x7f99d403a710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99d8006fd0 tx=0x7f99d8006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 msgr2=0x7f99e819c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4011a30 comp rx=0 tx=0).stop 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 shutdown_connections 2023-08-25T04:36:42.111 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99d4038250 0x7f99d403a710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 --2- 172.21.15.145:0/2419169456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99e807be30 0x7f99e819c0b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.701+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 >> 172.21.15.145:0/2419169456 conn(0x7f99e8101750 msgr2=0x7f99e8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.702+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 shutdown_connections 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr 2023-08-25T04:36:41.702+0000 7f99f0224700 1 -- 172.21.15.145:0/2419169456 wait complete. 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.112 INFO:teuthology.orchestra.run.smithi145.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout:Or, if you are only running a single cluster on this host: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout:Please consider enabling telemetry to help improve Ceph: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.113 INFO:teuthology.orchestra.run.smithi145.stdout: ceph telemetry on 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout:For more information see: 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:42.114 INFO:teuthology.orchestra.run.smithi145.stdout:Bootstrap complete. 2023-08-25T04:36:42.169 INFO:tasks.cephadm:Fetching config... 2023-08-25T04:36:42.169 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:36:42.170 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-08-25T04:36:42.216 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-08-25T04:36:42.217 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:36:42.218 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-08-25T04:36:42.301 INFO:tasks.cephadm:Fetching mon keyring... 2023-08-25T04:36:42.302 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:36:42.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-08-25T04:36:42.411 INFO:tasks.cephadm:Fetching pub ssh key... 2023-08-25T04:36:42.412 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:36:42.412 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-08-25T04:36:42.497 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-08-25T04:36:42.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T04:36:42.637 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:36:42.668 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T04:36:42.714 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:36:42.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-25T04:36:42.773 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSu1gFeGxO5PBGbLRpDX9gUQZ3+JQ2sKAfoy2eAcCuht1TNd1Obt9EZW209SLYGxYWz+QBebZkwlwIh1nIs9g030vn/hr7CQVSCvm64ph75w6QcUkSieW3dy3HBc8HAAP0r7Yv/0Z3lfa8r4FZ6dQBEar47rPMIBUYcSEiOenyNi+8VMX8w995BCN9cRPDykqyzSCz9zfl7axBW1w0/I/SEF4p6+SyBzS2GofiCs0D5hDYDOirBKSNcHiLTqgEu7Xpm9zQTs/gPnVv+kvaaUzYdstKxDZ9LalroQa0WltuBywn9qX1SU/7lUW/SLkmUmMzr4J7e7pBH8BanwktCp7x0Yth2hUw1X/p4/NJCASqBzgMimu6c/6CPuiVdGwJJjkw2PP0tsCPZcRA0L24m6nBDcApImrZO8s341cCtVvV/ocQvTLBzcvDfkul7vajAabq4zJ+2BhNBpvX2lds+X1k/nk5pC+Xvgztf1MSPUjMg+P+6MyxB5G/HvdgEvmyQKs= ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:36:42.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-08-25T04:36:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:42 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2419169456' entity='client.admin' 2023-08-25T04:36:43.727 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:36:45.003 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:45.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:45.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:36:45.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:45.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:45.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.261+0000 7efe7f521700 1 -- 172.21.15.145:0/1175695654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 msgr2=0x7efe78072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.261+0000 7efe7f521700 1 --2- 172.21.15.145:0/1175695654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe78072db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe74009910 tx=0x7efe74009c20 comp rx=0 tx=0).stop 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.261+0000 7efe7f521700 1 -- 172.21.15.145:0/1175695654 shutdown_connections 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.261+0000 7efe7f521700 1 --2- 172.21.15.145:0/1175695654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe78072db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.261+0000 7efe7f521700 1 -- 172.21.15.145:0/1175695654 >> 172.21.15.145:0/1175695654 conn(0x7efe780fb9c0 msgr2=0x7efe780fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:45.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.262+0000 7efe7f521700 1 -- 172.21.15.145:0/1175695654 shutdown_connections 2023-08-25T04:36:45.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.262+0000 7efe7f521700 1 -- 172.21.15.145:0/1175695654 wait complete. 2023-08-25T04:36:45.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7f521700 1 Processor -- start 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7f521700 1 -- start start 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7f521700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7f521700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7819c080 con 0x7efe78074950 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7d2bd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7d2bd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57806/0 (socket says 172.21.15.145:57806) 2023-08-25T04:36:45.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.263+0000 7efe7d2bd700 1 -- 172.21.15.145:0/2527919704 learned_addr learned my addr 172.21.15.145:0/2527919704 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe7d2bd700 1 -- 172.21.15.145:0/2527919704 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740095f0 con 0x7efe78074950 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe7d2bd700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe74006010 tx=0x7efe740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe7401d070 con 0x7efe78074950 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe740044e0 con 0x7efe78074950 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe740039b0 con 0x7efe78074950 2023-08-25T04:36:45.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7819c280 con 0x7efe78074950 2023-08-25T04:36:45.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.264+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7819c6c0 con 0x7efe78074950 2023-08-25T04:36:45.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.265+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7efe74003c70 con 0x7efe78074950 2023-08-25T04:36:45.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.265+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78195590 con 0x7efe78074950 2023-08-25T04:36:45.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.266+0000 7efe6e7fc700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 0x7efe6403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:45.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.266+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe7404c110 con 0x7efe78074950 2023-08-25T04:36:45.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.266+0000 7efe7cabc700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 0x7efe6403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:45.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.267+0000 7efe7cabc700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 0x7efe6403a670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:45.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.271+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7efe7402a540 con 0x7efe78074950 2023-08-25T04:36:45.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.480+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7efe7802cd60 con 0x7efe78074950 2023-08-25T04:36:45.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.483+0000 7efe6e7fc700 1 -- 172.21.15.145:0/2527919704 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7efe7402a360 con 0x7efe78074950 2023-08-25T04:36:45.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.486+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 msgr2=0x7efe6403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.486+0000 7efe7f521700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 0x7efe6403a670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.486+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 msgr2=0x7efe7819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.486+0000 7efe7f521700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe74006010 tx=0x7efe740057c0 comp rx=0 tx=0).stop 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 shutdown_connections 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efe640381b0 0x7efe6403a670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:45.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 --2- 172.21.15.145:0/2527919704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe78074950 0x7efe7819bb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:45.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 >> 172.21.15.145:0/2527919704 conn(0x7efe780fb9c0 msgr2=0x7efe780fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:45.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 shutdown_connections 2023-08-25T04:36:45.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:45.487+0000 7efe7f521700 1 -- 172.21.15.145:0/2527919704 wait complete. 2023-08-25T04:36:46.215 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-08-25T04:36:46.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-08-25T04:36:46.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2527919704' entity='client.admin' 2023-08-25T04:36:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:46.932 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:36:48.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.375+0000 7f0f29302700 1 -- 172.21.15.145:0/1370736931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 msgr2=0x7f0f24100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:48.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.375+0000 7f0f29302700 1 --2- 172.21.15.145:0/1370736931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f24100860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009910 tx=0x7f0f14009c20 comp rx=0 tx=0).stop 2023-08-25T04:36:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.376+0000 7f0f29302700 1 -- 172.21.15.145:0/1370736931 shutdown_connections 2023-08-25T04:36:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.376+0000 7f0f29302700 1 --2- 172.21.15.145:0/1370736931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f24100860 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.376+0000 7f0f29302700 1 -- 172.21.15.145:0/1370736931 >> 172.21.15.145:0/1370736931 conn(0x7f0f240fb9c0 msgr2=0x7f0f240fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.376+0000 7f0f29302700 1 -- 172.21.15.145:0/1370736931 shutdown_connections 2023-08-25T04:36:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.376+0000 7f0f29302700 1 -- 172.21.15.145:0/1370736931 wait complete. 2023-08-25T04:36:48.383 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f29302700 1 Processor -- start 2023-08-25T04:36:48.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f29302700 1 -- start start 2023-08-25T04:36:48.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f29302700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:48.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f29302700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24074c00 con 0x7f0f24100440 2023-08-25T04:36:48.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f22ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:48.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.377+0000 7f0f22ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57826/0 (socket says 172.21.15.145:57826) 2023-08-25T04:36:48.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f22ffd700 1 -- 172.21.15.145:0/1748935425 learned_addr learned my addr 172.21.15.145:0/1748935425 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:48.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f22ffd700 1 -- 172.21.15.145:0/1748935425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f140095f0 con 0x7f0f24100440 2023-08-25T04:36:48.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f22ffd700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f14004f40 tx=0x7f0f140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:48.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f1401d070 con 0x7f0f24100440 2023-08-25T04:36:48.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f24072d10 con 0x7f0f24100440 2023-08-25T04:36:48.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f1400bc10 con 0x7f0f24100440 2023-08-25T04:36:48.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f14003a70 con 0x7f0f24100440 2023-08-25T04:36:48.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.378+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f240731b0 con 0x7f0f24100440 2023-08-25T04:36:48.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.380+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f0f14003d30 con 0x7f0f24100440 2023-08-25T04:36:48.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.380+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f24061f50 con 0x7f0f24100440 2023-08-25T04:36:48.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.380+0000 7f0f0bfff700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 0x7f0f0c03a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:48.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.380+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f1404bd80 con 0x7f0f24100440 2023-08-25T04:36:48.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.380+0000 7f0f227fc700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 0x7f0f0c03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:48.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.381+0000 7f0f227fc700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 0x7f0f0c03a670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:48.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.385+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0f1402ae20 con 0x7f0f24100440 2023-08-25T04:36:48.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.585+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f2418ec60 con 0x7f0f0c0381b0 2023-08-25T04:36:48.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.589+0000 7f0f0bfff700 1 -- 172.21.15.145:0/1748935425 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0f2418ec60 con 0x7f0f0c0381b0 2023-08-25T04:36:48.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.591+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 msgr2=0x7f0f0c03a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.591+0000 7f0f29302700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 0x7f0f0c03a670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.591+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 msgr2=0x7f0f240746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.591+0000 7f0f29302700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f14004f40 tx=0x7f0f140057c0 comp rx=0 tx=0).stop 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 shutdown_connections 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0f0c0381b0 0x7f0f0c03a670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:48.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 --2- 172.21.15.145:0/1748935425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f24100440 0x7f0f240746c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:48.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 >> 172.21.15.145:0/1748935425 conn(0x7f0f240fb9c0 msgr2=0x7f0f240fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:48.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 shutdown_connections 2023-08-25T04:36:48.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:48.592+0000 7f0f29302700 1 -- 172.21.15.145:0/1748935425 wait complete. 2023-08-25T04:36:49.109 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi162 2023-08-25T04:36:49.109 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:36:49.109 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.conf 2023-08-25T04:36:49.130 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:36:49.130 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:36:49.190 INFO:tasks.cephadm:Adding host smithi162 to orchestrator... 2023-08-25T04:36:49.191 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch host add smithi162 2023-08-25T04:36:49.929 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:36:50.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:50.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:50.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:50.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:36:50.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:36:50.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:49 smithi145 ceph-mon[97681]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:36:50.788 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:50 smithi145 ceph-mon[97681]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:36:50.788 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:50 smithi145 ceph-mon[97681]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:36:51.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 -- 172.21.15.145:0/4214981269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 msgr2=0x7fb9580730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:51.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/4214981269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb9580730a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb948009910 tx=0x7fb948009c20 comp rx=0 tx=0).stop 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 -- 172.21.15.145:0/4214981269 shutdown_connections 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/4214981269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb9580730a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 -- 172.21.15.145:0/4214981269 >> 172.21.15.145:0/4214981269 conn(0x7fb9580fbf10 msgr2=0x7fb9580fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 -- 172.21.15.145:0/4214981269 shutdown_connections 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.389+0000 7fb95dcd2700 1 -- 172.21.15.145:0/4214981269 wait complete. 2023-08-25T04:36:51.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb95dcd2700 1 Processor -- start 2023-08-25T04:36:51.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb95dcd2700 1 -- start start 2023-08-25T04:36:51.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb95dcd2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:51.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb95dcd2700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95819c3a0 con 0x7fb958072cc0 2023-08-25T04:36:51.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:51.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57846/0 (socket says 172.21.15.145:57846) 2023-08-25T04:36:51.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.391+0000 7fb9577fe700 1 -- 172.21.15.145:0/3739866725 learned_addr learned my addr 172.21.15.145:0/3739866725 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb9577fe700 1 -- 172.21.15.145:0/3739866725 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480095f0 con 0x7fb958072cc0 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb9577fe700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb948006010 tx=0x7fb948004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb94801d070 con 0x7fb958072cc0 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9480044a0 con 0x7fb958072cc0 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95819c600 con 0x7fb958072cc0 2023-08-25T04:36:51.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9480039b0 con 0x7fb958072cc0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.392+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95819ca80 con 0x7fb958072cc0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.393+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958061f50 con 0x7fb958072cc0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.393+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7fb948003ba0 con 0x7fb958072cc0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.394+0000 7fb954ff9700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 0x7fb94003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.394+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb94804bee0 con 0x7fb958072cc0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.394+0000 7fb956ffd700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 0x7fb94003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:51.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.394+0000 7fb956ffd700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 0x7fb94003a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:51.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.399+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb948029360 con 0x7fb958072cc0 2023-08-25T04:36:51.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:51.601+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}) v1 -- 0x7fb958110e40 con 0x7fb940038200 2023-08-25T04:36:52.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:51 smithi145 ceph-mon[97681]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:36:52.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:52.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:52.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:53.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:52 smithi145 ceph-mon[97681]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:36:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.972+0000 7fb954ff9700 1 -- 172.21.15.145:0/3739866725 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb958110e40 con 0x7fb940038200 2023-08-25T04:36:53.973 INFO:teuthology.orchestra.run.smithi145.stdout:Added host 'smithi162' with addr '172.21.15.162' 2023-08-25T04:36:53.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 msgr2=0x7fb94003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:53.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 0x7fb94003a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:53.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 msgr2=0x7fb95819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:53.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb948006010 tx=0x7fb948004ff0 comp rx=0 tx=0).stop 2023-08-25T04:36:53.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 shutdown_connections 2023-08-25T04:36:53.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb940038200 0x7fb94003a6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:53.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 --2- 172.21.15.145:0/3739866725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb958072cc0 0x7fb95819be60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:53.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 >> 172.21.15.145:0/3739866725 conn(0x7fb9580fbf10 msgr2=0x7fb9580fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:53.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 shutdown_connections 2023-08-25T04:36:53.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:53.975+0000 7fb95dcd2700 1 -- 172.21.15.145:0/3739866725 wait complete. 2023-08-25T04:36:53.998 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:53 smithi145 ceph-mon[97681]: Deploying cephadm binary to smithi162 2023-08-25T04:36:54.411 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch host ls --format=json 2023-08-25T04:36:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:36:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:54 smithi145 ceph-mon[97681]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:36:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:54 smithi145 ceph-mon[97681]: Added host smithi162 2023-08-25T04:36:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:36:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:56 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:56.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 -- 172.21.15.145:0/509811654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 msgr2=0x7f91b8100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:56.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 --2- 172.21.15.145:0/509811654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b8100a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009910 tx=0x7f91b4009c20 comp rx=0 tx=0).stop 2023-08-25T04:36:56.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 -- 172.21.15.145:0/509811654 shutdown_connections 2023-08-25T04:36:56.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 --2- 172.21.15.145:0/509811654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b8100a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:56.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 -- 172.21.15.145:0/509811654 >> 172.21.15.145:0/509811654 conn(0x7f91b80fbf10 msgr2=0x7f91b80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:56.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 -- 172.21.15.145:0/509811654 shutdown_connections 2023-08-25T04:36:56.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.639+0000 7f91c0416700 1 -- 172.21.15.145:0/509811654 wait complete. 2023-08-25T04:36:56.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.640+0000 7f91c0416700 1 Processor -- start 2023-08-25T04:36:56.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91c0416700 1 -- start start 2023-08-25T04:36:56.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91c0416700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:56.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91c0416700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819dde0 con 0x7f91b8100620 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91be1b2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91be1b2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37996/0 (socket says 172.21.15.145:37996) 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.641+0000 7f91be1b2700 1 -- 172.21.15.145:0/647307822 learned_addr learned my addr 172.21.15.145:0/647307822 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91be1b2700 1 -- 172.21.15.145:0/647307822 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b40095f0 con 0x7f91b8100620 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91be1b2700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b401d070 con 0x7f91b8100620 2023-08-25T04:36:56.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b819dfe0 con 0x7f91b8100620 2023-08-25T04:36:56.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b400bc10 con 0x7f91b8100620 2023-08-25T04:36:56.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b4003910 con 0x7f91b8100620 2023-08-25T04:36:56.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.642+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b819e400 con 0x7f91b8100620 2023-08-25T04:36:56.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.643+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f91b400bd80 con 0x7f91b8100620 2023-08-25T04:36:56.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.643+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b8061f50 con 0x7f91b8100620 2023-08-25T04:36:56.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.644+0000 7f91af7fe700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 0x7f91a403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:56.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.644+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91b404bd60 con 0x7f91b8100620 2023-08-25T04:36:56.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.644+0000 7f91bd9b1700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 0x7f91a403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:56.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.644+0000 7f91bd9b1700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 0x7f91a403a670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:56.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.649+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f91b4029360 con 0x7f91b8100620 2023-08-25T04:36:56.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.852+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91b810a970 con 0x7f91a40381b0 2023-08-25T04:36:56.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.854+0000 7f91af7fe700 1 -- 172.21.15.145:0/647307822 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f91b810a970 con 0x7f91a40381b0 2023-08-25T04:36:56.855 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:36:56.855 INFO:teuthology.orchestra.run.smithi145.stdout:[{"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}] 2023-08-25T04:36:56.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 msgr2=0x7f91a403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:56.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 0x7f91a403a670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).stop 2023-08-25T04:36:56.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 msgr2=0x7f91b819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b4004ff0 comp rx=0 tx=0).stop 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 shutdown_connections 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f91a40381b0 0x7f91a403a670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 --2- 172.21.15.145:0/647307822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f91b8100620 0x7f91b819d8a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 >> 172.21.15.145:0/647307822 conn(0x7f91b80fbf10 msgr2=0x7f91b8104000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 shutdown_connections 2023-08-25T04:36:56.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:56.856+0000 7f91c0416700 1 -- 172.21.15.145:0/647307822 wait complete. 2023-08-25T04:36:57.293 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:57 smithi145 ceph-mon[97681]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:36:57.294 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:36:57.308 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2023-08-25T04:36:57.308 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:36:57.309 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2023-08-25T04:36:57.332 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:36:57.332 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:36:57.390 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2023-08-25T04:36:57.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch host add smithi183 2023-08-25T04:36:58.104 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:36:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:58 smithi145 ceph-mon[97681]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:36:59.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:36:59 smithi145 ceph-mon[97681]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:36:59.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.608+0000 7fc64de0b700 1 -- 172.21.15.145:0/1482859202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 msgr2=0x7fc648100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:36:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.608+0000 7fc64de0b700 1 --2- 172.21.15.145:0/1482859202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc648100a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc638009910 tx=0x7fc638009c20 comp rx=0 tx=0).stop 2023-08-25T04:36:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.609+0000 7fc64de0b700 1 -- 172.21.15.145:0/1482859202 shutdown_connections 2023-08-25T04:36:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.609+0000 7fc64de0b700 1 --2- 172.21.15.145:0/1482859202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc648100a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:36:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.609+0000 7fc64de0b700 1 -- 172.21.15.145:0/1482859202 >> 172.21.15.145:0/1482859202 conn(0x7fc6480fbf10 msgr2=0x7fc6480fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:36:59.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.609+0000 7fc64de0b700 1 -- 172.21.15.145:0/1482859202 shutdown_connections 2023-08-25T04:36:59.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.609+0000 7fc64de0b700 1 -- 172.21.15.145:0/1482859202 wait complete. 2023-08-25T04:36:59.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc64de0b700 1 Processor -- start 2023-08-25T04:36:59.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc64de0b700 1 -- start start 2023-08-25T04:36:59.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc64de0b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:59.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc64de0b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64819dea0 con 0x7fc648100620 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc6477fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc6477fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38024/0 (socket says 172.21.15.145:38024) 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.610+0000 7fc6477fe700 1 -- 172.21.15.145:0/733310987 learned_addr learned my addr 172.21.15.145:0/733310987 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc6477fe700 1 -- 172.21.15.145:0/733310987 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380095f0 con 0x7fc648100620 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc6477fe700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc638006010 tx=0x7fc638004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc63801d070 con 0x7fc648100620 2023-08-25T04:36:59.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63800bc10 con 0x7fc648100620 2023-08-25T04:36:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc638003910 con 0x7fc648100620 2023-08-25T04:36:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64819e0a0 con 0x7fc648100620 2023-08-25T04:36:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.611+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc648197950 con 0x7fc648100620 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.612+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7fc63800bd80 con 0x7fc648100620 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.612+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64819e4c0 con 0x7fc648100620 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.613+0000 7fc644ff9700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 0x7fc63003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.613+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc63804bdc0 con 0x7fc648100620 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.613+0000 7fc646ffd700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 0x7fc63003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:36:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.614+0000 7fc646ffd700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 0x7fc63003a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:36:59.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.618+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc638029360 con 0x7fc648100620 2023-08-25T04:36:59.828 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:36:59.827+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) v1 -- 0x7fc64810a780 con 0x7fc630038200 2023-08-25T04:37:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:00.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:00.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:00.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:01.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:01 smithi145 ceph-mon[97681]: Deploying cephadm binary to smithi183 2023-08-25T04:37:02.150 INFO:teuthology.orchestra.run.smithi145.stdout:Added host 'smithi183' with addr '172.21.15.183' 2023-08-25T04:37:02.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.149+0000 7fc644ff9700 1 -- 172.21.15.145:0/733310987 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc64810a780 con 0x7fc630038200 2023-08-25T04:37:02.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.151+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 msgr2=0x7fc63003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:02.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.151+0000 7fc64de0b700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 0x7fc63003a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c006e40 comp rx=0 tx=0).stop 2023-08-25T04:37:02.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.151+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 msgr2=0x7fc64819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:02.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.151+0000 7fc64de0b700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc638006010 tx=0x7fc638004ff0 comp rx=0 tx=0).stop 2023-08-25T04:37:02.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.151+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 shutdown_connections 2023-08-25T04:37:02.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.152+0000 7fc64de0b700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc630038200 0x7fc63003a6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:02.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.152+0000 7fc64de0b700 1 --2- 172.21.15.145:0/733310987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc648100620 0x7fc64819d960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:02.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.152+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 >> 172.21.15.145:0/733310987 conn(0x7fc6480fbf10 msgr2=0x7fc648104000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:02.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.152+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 shutdown_connections 2023-08-25T04:37:02.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:02.152+0000 7fc64de0b700 1 -- 172.21.15.145:0/733310987 wait complete. 2023-08-25T04:37:02.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch host ls --format=json 2023-08-25T04:37:02.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:02 smithi145 ceph-mon[97681]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:03.307 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:37:03.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:03 smithi145 ceph-mon[97681]: Added host smithi183 2023-08-25T04:37:03.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:03.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:04.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.771+0000 7eff21775700 1 -- 172.21.15.145:0/3850241570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 msgr2=0x7eff1c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:04.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.771+0000 7eff21775700 1 --2- 172.21.15.145:0/3850241570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c100a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009910 tx=0x7eff0c009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:04.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.771+0000 7eff21775700 1 -- 172.21.15.145:0/3850241570 shutdown_connections 2023-08-25T04:37:04.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.771+0000 7eff21775700 1 --2- 172.21.15.145:0/3850241570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c100a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:04.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.771+0000 7eff21775700 1 -- 172.21.15.145:0/3850241570 >> 172.21.15.145:0/3850241570 conn(0x7eff1c0fbf10 msgr2=0x7eff1c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.772+0000 7eff21775700 1 -- 172.21.15.145:0/3850241570 shutdown_connections 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.772+0000 7eff21775700 1 -- 172.21.15.145:0/3850241570 wait complete. 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff21775700 1 Processor -- start 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff21775700 1 -- start start 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff21775700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff21775700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19dde0 con 0x7eff1c100620 2023-08-25T04:37:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff1affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff1affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42722/0 (socket says 172.21.15.145:42722) 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.773+0000 7eff1affd700 1 -- 172.21.15.145:0/2064368972 learned_addr learned my addr 172.21.15.145:0/2064368972 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff1affd700 1 -- 172.21.15.145:0/2064368972 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0095f0 con 0x7eff1c100620 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff1affd700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006010 tx=0x7eff0c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff0c01d070 con 0x7eff1c100620 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0c00bc10 con 0x7eff1c100620 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c19dfe0 con 0x7eff1c100620 2023-08-25T04:37:04.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff0c003910 con 0x7eff1c100620 2023-08-25T04:37:04.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.774+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c19e400 con 0x7eff1c100620 2023-08-25T04:37:04.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.775+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7eff0c003b80 con 0x7eff1c100620 2023-08-25T04:37:04.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.775+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c061f50 con 0x7eff1c100620 2023-08-25T04:37:04.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.776+0000 7eff03fff700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 0x7eff0403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:04.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.776+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff0c04be00 con 0x7eff1c100620 2023-08-25T04:37:04.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.776+0000 7eff1a7fc700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 0x7eff0403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:04.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.776+0000 7eff1a7fc700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 0x7eff0403a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff10006fd0 tx=0x7eff10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:04.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.781+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7eff0c00ba70 con 0x7eff1c100620 2023-08-25T04:37:04.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.982+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff1c10a930 con 0x7eff04038200 2023-08-25T04:37:04.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.983+0000 7eff03fff700 1 -- 172.21.15.145:0/2064368972 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7eff1c10a930 con 0x7eff04038200 2023-08-25T04:37:04.984 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:37:04.984 INFO:teuthology.orchestra.run.smithi145.stdout:[{"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2023-08-25T04:37:04.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 msgr2=0x7eff0403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:04.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 0x7eff0403a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff10006fd0 tx=0x7eff10006e40 comp rx=0 tx=0).stop 2023-08-25T04:37:04.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 msgr2=0x7eff1c19d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:04.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006010 tx=0x7eff0c004ff0 comp rx=0 tx=0).stop 2023-08-25T04:37:04.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 shutdown_connections 2023-08-25T04:37:04.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7eff04038200 0x7eff0403a6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:04.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 --2- 172.21.15.145:0/2064368972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff1c100620 0x7eff1c19d8a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:04.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 >> 172.21.15.145:0/2064368972 conn(0x7eff1c0fbf10 msgr2=0x7eff1c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:04.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 shutdown_connections 2023-08-25T04:37:04.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:04.986+0000 7eff21775700 1 -- 172.21.15.145:0/2064368972 wait complete. 2023-08-25T04:37:05.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:05 smithi145 ceph-mon[97681]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:05.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:05 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:05.476 INFO:tasks.cephadm:Setting crush tunables to default 2023-08-25T04:37:05.477 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd crush tunables default 2023-08-25T04:37:06.196 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:37:06.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:06 smithi145 ceph-mon[97681]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:37:07.144 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:07.144 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:07.144 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:07.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:07.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:07.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:07.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:07.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:07 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 -- 172.21.15.145:0/1906483996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 msgr2=0x7f0688100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 --2- 172.21.15.145:0/1906483996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f0688100a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0684009910 tx=0x7f0684009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 -- 172.21.15.145:0/1906483996 shutdown_connections 2023-08-25T04:37:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 --2- 172.21.15.145:0/1906483996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f0688100a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 -- 172.21.15.145:0/1906483996 >> 172.21.15.145:0/1906483996 conn(0x7f06880fbf10 msgr2=0x7f06880fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 -- 172.21.15.145:0/1906483996 shutdown_connections 2023-08-25T04:37:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.764+0000 7f0690164700 1 -- 172.21.15.145:0/1906483996 wait complete. 2023-08-25T04:37:07.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.765+0000 7f0690164700 1 Processor -- start 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f0690164700 1 -- start start 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f0690164700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f0690164700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068819dd90 con 0x7f0688100620 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f068df00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f068df00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42738/0 (socket says 172.21.15.145:42738) 2023-08-25T04:37:07.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f068df00700 1 -- 172.21.15.145:0/3223805074 learned_addr learned my addr 172.21.15.145:0/3223805074 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:37:07.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.766+0000 7f068df00700 1 -- 172.21.15.145:0/3223805074 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840095f0 con 0x7f0688100620 2023-08-25T04:37:07.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f068df00700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0684004f40 tx=0x7f0684005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:07.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f068401d070 con 0x7f0688100620 2023-08-25T04:37:07.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068819df90 con 0x7f0688100620 2023-08-25T04:37:07.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0684004370 con 0x7f0688100620 2023-08-25T04:37:07.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06840039b0 con 0x7f0688100620 2023-08-25T04:37:07.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.767+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068819e3b0 con 0x7f0688100620 2023-08-25T04:37:07.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.768+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f0684003c70 con 0x7f0688100620 2023-08-25T04:37:07.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.768+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0688061f50 con 0x7f0688100620 2023-08-25T04:37:07.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.769+0000 7f067effd700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 0x7f067403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:07.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.769+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f068404c0b0 con 0x7f0688100620 2023-08-25T04:37:07.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.769+0000 7f068d6ff700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 0x7f067403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:07.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.769+0000 7f068d6ff700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 0x7f067403a670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:07.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.773+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0684029360 con 0x7f0688100620 2023-08-25T04:37:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:07.980+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0688198de0 con 0x7f0688100620 2023-08-25T04:37:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:08.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T04:37:08.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.433+0000 7f067effd700 1 -- 172.21.15.145:0/3223805074 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0688198de0 con 0x7f0688100620 2023-08-25T04:37:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 msgr2=0x7f067403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 0x7f067403a670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).stop 2023-08-25T04:37:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 msgr2=0x7f068819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0684004f40 tx=0x7f0684005020 comp rx=0 tx=0).stop 2023-08-25T04:37:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 shutdown_connections 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f06740381b0 0x7f067403a670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 --2- 172.21.15.145:0/3223805074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0688100620 0x7f068819d850 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 >> 172.21.15.145:0/3223805074 conn(0x7f06880fbf10 msgr2=0x7f0688102880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 shutdown_connections 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:08.436+0000 7f0690164700 1 -- 172.21.15.145:0/3223805074 wait complete. 2023-08-25T04:37:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:adjusted tunables profile to default 2023-08-25T04:37:08.874 INFO:tasks.cephadm:Adding mon.a on smithi145 2023-08-25T04:37:08.874 INFO:tasks.cephadm:Adding mon.b on smithi162 2023-08-25T04:37:08.875 INFO:tasks.cephadm:Adding mon.c on smithi183 2023-08-25T04:37:08.875 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch apply mon '3;smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c' 2023-08-25T04:37:09.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:09 smithi145 ceph-mon[97681]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:09 smithi145 ceph-mon[97681]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:09 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T04:37:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:09 smithi145 ceph-mon[97681]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:09.462 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:10.556 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/288292504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 msgr2=0x7f7ec0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:10.557 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/288292504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0100030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009910 tx=0x7f7eb0009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:10.557 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/288292504 shutdown_connections 2023-08-25T04:37:10.557 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/288292504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec0100030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:10.557 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/288292504 >> 172.21.15.183:0/288292504 conn(0x7f7ec0075a10 msgr2=0x7f7ec0075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/288292504 shutdown_connections 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.554+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/288292504 wait complete. 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec7dd5700 1 Processor -- start 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec7dd5700 1 -- start start 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec7dd5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:10.558 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec7dd5700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ff7e0 con 0x7f7ec00ffc50 2023-08-25T04:37:10.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec5b71700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:10.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec5b71700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.183:36908/0 (socket says 172.21.15.183:36908) 2023-08-25T04:37:10.559 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec5b71700 1 -- 172.21.15.183:0/3469748579 learned_addr learned my addr 172.21.15.183:0/3469748579 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:37:10.560 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.555+0000 7f7ec5b71700 1 -- 172.21.15.183:0/3469748579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00095f0 con 0x7f7ec00ffc50 2023-08-25T04:37:10.562 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7ec5b71700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006010 tx=0x7f7eb0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:10.562 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb001d070 con 0x7f7ec00ffc50 2023-08-25T04:37:10.562 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0194a30 con 0x7f7ec00ffc50 2023-08-25T04:37:10.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eb000bc10 con 0x7f7ec00ffc50 2023-08-25T04:37:10.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb0003910 con 0x7f7ec00ffc50 2023-08-25T04:37:10.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.556+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0194eb0 con 0x7f7ec00ffc50 2023-08-25T04:37:10.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.557+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec01036a0 con 0x7f7ec00ffc50 2023-08-25T04:37:10.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.557+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f7eb0003bd0 con 0x7f7ec00ffc50 2023-08-25T04:37:10.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.557+0000 7f7eb6ffd700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 0x7f7eac03a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:10.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.557+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7eb004be50 con 0x7f7ec00ffc50 2023-08-25T04:37:10.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.557+0000 7f7ec5370700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 0x7f7eac03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:10.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.558+0000 7f7ec5370700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 0x7f7eac03a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc006fd0 tx=0x7f7ebc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:10.565 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.563+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7eb000ba70 con 0x7f7ec00ffc50 2023-08-25T04:37:10.761 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.759+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec0195160 con 0x7f7eac0381b0 2023-08-25T04:37:10.768 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.767+0000 7f7eb6ffd700 1 -- 172.21.15.183:0/3469748579 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7ec0195160 con 0x7f7eac0381b0 2023-08-25T04:37:10.768 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2023-08-25T04:37:10.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 msgr2=0x7f7eac03a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:10.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 0x7f7eac03a670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc006fd0 tx=0x7f7ebc006e40 comp rx=0 tx=0).stop 2023-08-25T04:37:10.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 msgr2=0x7f7ec01944f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:10.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006010 tx=0x7f7eb0004ff0 comp rx=0 tx=0).stop 2023-08-25T04:37:10.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 shutdown_connections 2023-08-25T04:37:10.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7eac0381b0 0x7f7eac03a670 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:10.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 --2- 172.21.15.183:0/3469748579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ec00ffc50 0x7f7ec01944f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:10.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 >> 172.21.15.183:0/3469748579 conn(0x7f7ec0075a10 msgr2=0x7f7ec00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:10.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 shutdown_connections 2023-08-25T04:37:10.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:10.768+0000 7f7ec7dd5700 1 -- 172.21.15.183:0/3469748579 wait complete. 2023-08-25T04:37:10.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:10 smithi145 ceph-mon[97681]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:11.141 DEBUG:teuthology.orchestra.run.smithi162:mon.b> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.b.service 2023-08-25T04:37:11.144 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.c.service 2023-08-25T04:37:11.146 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T04:37:11.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T04:37:11.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:37:11.181 INFO:journalctl@ceph.mon.b.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:37:11.909 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: Saving service mon spec with placement smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c;count:3 2023-08-25T04:37:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:11 smithi145 ceph-mon[97681]: Deploying daemon mon.c on smithi183 2023-08-25T04:37:13.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:12 smithi145 ceph-mon[97681]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:13.527 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 -- 172.21.15.183:0/2013927030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 msgr2=0x7f2c800ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:13.527 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 --2- 172.21.15.183:0/2013927030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c800ff8e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009910 tx=0x7f2c7c009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:13.528 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 -- 172.21.15.183:0/2013927030 shutdown_connections 2023-08-25T04:37:13.528 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 --2- 172.21.15.183:0/2013927030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c800ff8e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:13.529 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 -- 172.21.15.183:0/2013927030 >> 172.21.15.183:0/2013927030 conn(0x7f2c800fadb0 msgr2=0x7f2c800fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:13.529 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 -- 172.21.15.183:0/2013927030 shutdown_connections 2023-08-25T04:37:13.529 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.524+0000 7f2c87608700 1 -- 172.21.15.183:0/2013927030 wait complete. 2023-08-25T04:37:13.529 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c87608700 1 Processor -- start 2023-08-25T04:37:13.530 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c87608700 1 -- start start 2023-08-25T04:37:13.530 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c87608700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:13.530 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c87608700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80196c60 con 0x7f2c800ff500 2023-08-25T04:37:13.531 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c853a4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:13.531 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c853a4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.183:36914/0 (socket says 172.21.15.183:36914) 2023-08-25T04:37:13.531 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.525+0000 7f2c853a4700 1 -- 172.21.15.183:0/3869683588 learned_addr learned my addr 172.21.15.183:0/3869683588 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:37:13.531 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c853a4700 1 -- 172.21.15.183:0/3869683588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0095f0 con 0x7f2c800ff500 2023-08-25T04:37:13.532 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c853a4700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c004f40 tx=0x7f2c7c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:13.532 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c7c01d070 con 0x7f2c800ff500 2023-08-25T04:37:13.533 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c7c004370 con 0x7f2c800ff500 2023-08-25T04:37:13.533 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c7c0039b0 con 0x7f2c800ff500 2023-08-25T04:37:13.534 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c87608700 1 -- 172.21.15.183:0/3869683588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80196e60 con 0x7f2c800ff500 2023-08-25T04:37:13.534 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.526+0000 7f2c87608700 1 -- 172.21.15.183:0/3869683588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80197280 con 0x7f2c800ff500 2023-08-25T04:37:13.534 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.529+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80197580 con 0x7f2c800ff500 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.534+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f2c7c003c70 con 0x7f2c800ff500 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.534+0000 7f2c767fc700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 0x7f2c6c03a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.534+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c7c04bda0 con 0x7f2c800ff500 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.536+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2c7c015350 con 0x7f2c800ff500 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.537+0000 7f2c84ba3700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 0x7f2c6c03a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:13.539 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.537+0000 7f2c84ba3700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 0x7f2c6c03a6c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c70006fd0 tx=0x7f2c70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:13.830 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.828+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c80061f50 con 0x7f2c800ff500 2023-08-25T04:37:13.831 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.828+0000 7f2c767fc700 1 -- 172.21.15.183:0/3869683588 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f2c7c026070 con 0x7f2c800ff500 2023-08-25T04:37:13.831 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:37:13.831 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","modified":"2023-08-25T04:35:29.770551Z","created":"2023-08-25T04:35:29.770551Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-08-25T04:37:13.832 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 msgr2=0x7f2c6c03a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:13.832 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 0x7f2c6c03a6c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c70006fd0 tx=0x7f2c70006e40 comp rx=0 tx=0).stop 2023-08-25T04:37:13.832 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 msgr2=0x7f2c80196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c004f40 tx=0x7f2c7c005020 comp rx=0 tx=0).stop 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 shutdown_connections 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2c6c038200 0x7f2c6c03a6c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 --2- 172.21.15.183:0/3869683588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c800ff500 0x7f2c80196720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 >> 172.21.15.183:0/3869683588 conn(0x7f2c800fadb0 msgr2=0x7f2c80103770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 shutdown_connections 2023-08-25T04:37:13.833 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:13.831+0000 7f2c6bfff700 1 -- 172.21.15.183:0/3869683588 wait complete. 2023-08-25T04:37:13.834 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2023-08-25T04:37:14.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:13 smithi183 podman[131354]: 2023-08-25T04:37:14.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 bash[131354]: a4c7df1ffeca8c58760c7f81af2afe9076f3cdaeea927d08de6fb1a85b3204a8 2023-08-25T04:37:14.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T04:37:14.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T04:37:14.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pidfile_write: ignore empty --pid-file 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: load: jerasure load: lrc 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: RocksDB version: 7.9.2 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Git sha 0 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: DB SUMMARY 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: DB Session ID: 0LWYBE9ZB1BNDZBJ82QC 2023-08-25T04:37:14.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: CURRENT file: CURRENT 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.error_if_exists: 0 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.create_if_missing: 0 2023-08-25T04:37:14.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.paranoid_checks: 1 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.env: 0x55ff61b57da0 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.info_log: 0x55ff63b7f440 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T04:37:14.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.statistics: (nil) 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.use_fsync: 0 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_log_file_size: 0 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_fallocate: 1 2023-08-25T04:37:14.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.use_direct_reads: 0 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.db_log_dir: 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.wal_dir: 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T04:37:14.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T04:37:14.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T04:37:14.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T04:37:14.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T04:37:14.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T04:37:14.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.write_buffer_manager: 0x55ff63b394a0 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T04:37:14.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.unordered_write: 0 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T04:37:14.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.row_cache: None 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.wal_filter: None 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.two_write_queues: 0 2023-08-25T04:37:14.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.wal_compression: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.atomic_flush: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.log_readahead_size: 0 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T04:37:14.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T04:37:14.684 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T04:37:14.684 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T04:37:14.684 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T04:37:14.685 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_background_jobs: 2 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_background_compactions: -1 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_subcompactions: 1 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T04:37:14.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_open_files: -1 2023-08-25T04:37:14.687 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_background_flushes: -1 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Compression algorithms supported: 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T04:37:14.688 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kZSTD supported: 0 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kXpressCompression supported: 0 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kZlibCompression supported: 1 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kSnappyCompression supported: 1 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kLZ4Compression supported: 1 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: kBZip2Compression supported: 0 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T04:37:14.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.merge_operator: 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_filter: None 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T04:37:14.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ff63b7f540) 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2023-08-25T04:37:14.691 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x55ff63bd71f0 2023-08-25T04:37:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2023-08-25T04:37:14.693 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2023-08-25T04:37:14.694 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2023-08-25T04:37:14.695 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression: NoCompression 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T04:37:14.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.num_levels: 7 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T04:37:14.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T04:37:14.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T04:37:14.699 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T04:37:14.700 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T04:37:14.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T04:37:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T04:37:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.table_properties_collectors: 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.inplace_update_support: 0 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.bloom_locality: 0 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.max_successive_merges: 0 2023-08-25T04:37:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.ttl: 2592000 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T04:37:14.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enable_blob_files: false 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.min_blob_size: 0 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T04:37:14.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 43a3a880-c013-44a3-b943-d2372d272be6 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938234291014, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-25T04:37:14.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938234291429, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692938234, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "43a3a880-c013-44a3-b943-d2372d272be6", "db_session_id": "0LWYBE9ZB1BNDZBJ82QC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938234291497, "job": 1, "event": "recovery_finished"} 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ff63c74000 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: DB pointer 0x55ff63c60000 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T04:37:14.708 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:14.709 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:14.710 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.711 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:37:14.712 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x55ff63bd71f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:14.713 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(???) e0 preinit fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 systemd[1]: Started Ceph mon.c for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).mds e1 new map 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).mds e1 print_map 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T04:37:14.714 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-25T04:37:14.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Added host smithi162 2023-08-25T04:37:14.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.717 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:14.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Deploying cephadm binary to smithi183 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Added host smithi183 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:14.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.722 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Saving service mon spec with placement smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c;count:3 2023-08-25T04:37:14.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:14.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: Deploying daemon mon.c on smithi183 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expand map: {default=false} 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta from 'false' to 'false' 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expanded map: {default=false} 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expand map: {default=info} 2023-08-25T04:37:14.725 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta from 'info' to 'info' 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expanded map: {default=info} 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expand map: {default=daemon} 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expand map: {default=debug} 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T04:37:14.726 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:14 smithi183 ceph-mon[131414]: expand_channel_meta expanded map: {default=debug} 2023-08-25T04:37:15.345 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T04:37:15.346 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T04:37:15.910 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:37:19.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:19.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: mon.a calling monitor election 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: mon.c calling monitor election 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: monmap e2: 2 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: fsmap 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:19.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: mgrmap e13: x(active, since 45s) 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: overall HEALTH_OK 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:19.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:19 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:19.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:19.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: mon.a calling monitor election 2023-08-25T04:37:19.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: mon.c calling monitor election 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: monmap e2: 2 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: fsmap 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: mgrmap e13: x(active, since 45s) 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: overall HEALTH_OK 2023-08-25T04:37:19.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:19.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:19.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:19 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:20.132 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 -- 172.21.15.183:0/882734756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 msgr2=0x7f1ce0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:20.133 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 --2- 172.21.15.183:0/882734756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1ce0005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4076310 tx=0x7f1ce400feb0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.133 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 -- 172.21.15.183:0/882734756 shutdown_connections 2023-08-25T04:37:20.133 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 --2- 172.21.15.183:0/882734756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1ce0005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.133 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 -- 172.21.15.183:0/882734756 >> 172.21.15.183:0/882734756 conn(0x7f1cf4074ed0 msgr2=0x7f1cf40752e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:20.134 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 -- 172.21.15.183:0/882734756 shutdown_connections 2023-08-25T04:37:20.134 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.131+0000 7f1cfb337700 1 -- 172.21.15.183:0/882734756 wait complete. 2023-08-25T04:37:20.135 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 Processor -- start 2023-08-25T04:37:20.135 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 -- start start 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1cf419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf419e480 con 0x7f1ce00035c0 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cfb337700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf419e5f0 con 0x7f1cf419df40 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:20.136 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52066/0 (socket says 172.21.15.183:52066) 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 -- 172.21.15.183:0/2451695180 learned_addr learned my addr 172.21.15.183:0/2451695180 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 -- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 msgr2=0x7f1cf4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 -- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 msgr2=0x7f1cf4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.132+0000 7f1cf88d2700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf90d3700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1cf419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf88d2700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf90d3700 1 -- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 msgr2=0x7f1cf4197a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:37:20.137 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf90d3700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf90d3700 1 -- 172.21.15.183:0/2451695180 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce400f900 con 0x7f1ce00035c0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cf90d3700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1cf419da00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1ce4013b00 con 0x7f1ce00035c0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce40044f0 con 0x7f1ce00035c0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf4198080 con 0x7f1ce00035c0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.133+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1ce4003d30 con 0x7f1ce00035c0 2023-08-25T04:37:20.138 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.134+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf41985d0 con 0x7f1ce00035c0 2023-08-25T04:37:20.139 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.134+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f1ce4011470 con 0x7f1ce00035c0 2023-08-25T04:37:20.139 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.135+0000 7f1cea7fc700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 0x7f1cd803a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:20.139 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.135+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ce404d680 con 0x7f1ce00035c0 2023-08-25T04:37:20.140 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.135+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf4061f50 con 0x7f1ce00035c0 2023-08-25T04:37:20.140 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.135+0000 7f1cf88d2700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 0x7f1cd803a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:20.140 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.136+0000 7f1cf88d2700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 0x7f1cd803a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009180 tx=0x7f1cf0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:20.142 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.142+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1ce4023d00 con 0x7f1ce00035c0 2023-08-25T04:37:20.405 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.404+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1cf4107120 con 0x7f1ce00035c0 2023-08-25T04:37:20.406 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.406+0000 7f1cea7fc700 1 -- 172.21.15.183:0/2451695180 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f1ce4023d00 con 0x7f1ce00035c0 2023-08-25T04:37:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:37:20.407 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","modified":"2023-08-25T04:37:14.336284Z","created":"2023-08-25T04:35:29.770551Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-25T04:37:20.408 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 msgr2=0x7f1cd803a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:20.408 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 0x7f1cd803a4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009180 tx=0x7f1cf0008040 comp rx=0 tx=0).stop 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 msgr2=0x7f1cf419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1cf419da00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce40047a0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 shutdown_connections 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1cd8037ff0 0x7f1cd803a4b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ce00035c0 0x7f1cf419da00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 --2- 172.21.15.183:0/2451695180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf419df40 0x7f1cf4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:20.409 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 >> 172.21.15.183:0/2451695180 conn(0x7f1cf4074ed0 msgr2=0x7f1cf4190c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:20.410 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 shutdown_connections 2023-08-25T04:37:20.410 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:20.407+0000 7f1cfb337700 1 -- 172.21.15.183:0/2451695180 wait complete. 2023-08-25T04:37:20.410 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2023-08-25T04:37:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:20 smithi183 ceph-mon[131414]: Deploying daemon mon.b on smithi162 2023-08-25T04:37:20.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:20 smithi183 ceph-mon[131414]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:20.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:20.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:20 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/2451695180' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:20 smithi145 ceph-mon[97681]: Deploying daemon mon.b on smithi162 2023-08-25T04:37:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:20 smithi145 ceph-mon[97681]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:20.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:20 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/2451695180' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:21.762 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-25T04:37:21.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph mon dump -f json 2023-08-25T04:37:21.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:37:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:37:21.336+0000 7f1bc5793700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-25T04:37:21.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:21 smithi162 systemd[1]: Starting Ceph mon.b for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:37:22.324 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:37:22.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 podman[130901]: 2023-08-25T04:37:22.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: set uid:gid to 167:167 (ceph:ceph) 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev), process ceph-mon, pid 7 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pidfile_write: ignore empty --pid-file 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 bash[130901]: e0b99022b4002dc6ae770326e422d825e35e5f6a7f9bcf69d7ed0c8e209b2b7d 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: load: jerasure load: lrc 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: RocksDB version: 7.9.2 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Git sha 0 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Compile date 2023-08-24 22:08:31 2023-08-25T04:37:22.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: DB SUMMARY 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: DB Session ID: UM9Q61X9OZG9DCBEEU46 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: CURRENT file: CURRENT 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: IDENTITY file: IDENTITY 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-08-25T04:37:22.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.error_if_exists: 0 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.create_if_missing: 0 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.paranoid_checks: 1 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.env: 0x55e58c1a0da0 2023-08-25T04:37:22.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.fs: PosixFileSystem 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.info_log: 0x55e58e4f3440 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_file_opening_threads: 16 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.statistics: (nil) 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.use_fsync: 0 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_log_file_size: 0 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-25T04:37:22.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.keep_log_file_num: 1000 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.recycle_log_file_num: 0 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_fallocate: 1 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_mmap_reads: 0 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_mmap_writes: 0 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.use_direct_reads: 0 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-25T04:37:22.417 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.create_missing_column_families: 0 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.db_log_dir: 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.wal_dir: 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-25T04:37:22.418 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-25T04:37:22.419 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.advise_random_on_open: 1 2023-08-25T04:37:22.419 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.db_write_buffer_size: 0 2023-08-25T04:37:22.419 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.write_buffer_manager: 0x55e58e4ad4a0 2023-08-25T04:37:22.419 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-25T04:37:22.419 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.rate_limiter: (nil) 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.wal_recovery_mode: 2 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enable_thread_tracking: 0 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enable_pipelined_write: 0 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.unordered_write: 0 2023-08-25T04:37:22.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.row_cache: None 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.wal_filter: None 2023-08-25T04:37:22.421 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_ingest_behind: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.two_write_queues: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.manual_wal_flush: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.wal_compression: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.atomic_flush: 0 2023-08-25T04:37:22.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.log_readahead_size: 0 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.best_efforts_recovery: 0 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-25T04:37:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.allow_data_in_errors: 0 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.db_host_id: __hostname__ 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_background_jobs: 2 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_background_compactions: -1 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_subcompactions: 1 2023-08-25T04:37:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_total_wal_size: 0 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-25T04:37:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_open_files: -1 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bytes_per_sync: 0 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_readahead_size: 0 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_background_flushes: -1 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Compression algorithms supported: 2023-08-25T04:37:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kZSTD supported: 0 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kXpressCompression supported: 0 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kLZ4HCCompression supported: 1 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kZlibCompression supported: 1 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kSnappyCompression supported: 1 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kLZ4Compression supported: 1 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: kBZip2Compression supported: 0 2023-08-25T04:37:22.427 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.merge_operator: 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_filter: None 2023-08-25T04:37:22.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_filter_factory: None 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.sst_partitioner_factory: None 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e58e4f3540) 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout: cache_index_and_filter_blocks: 1 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-25T04:37:22.429 INFO:journalctl@ceph.mon.b.smithi162.stdout: pin_top_level_index_and_filter: 1 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_type: 0 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: data_block_index_type: 0 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_shortening: 1 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: checksum: 4 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: no_block_cache: 0 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache: 0x55e58e54b1f0 2023-08-25T04:37:22.430 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_name: BinnedLRUCache 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_options: 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: capacity : 536870912 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: num_shard_bits : 4 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: strict_capacity_limit : 0 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: high_pri_pool_ratio: 0.000 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_compressed: (nil) 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: persistent_cache: (nil) 2023-08-25T04:37:22.431 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_size: 4096 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_size_deviation: 10 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_restart_interval: 16 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_block_restart_interval: 1 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: metadata_block_size: 4096 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: partition_filters: 0 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: use_delta_encoding: 1 2023-08-25T04:37:22.432 INFO:journalctl@ceph.mon.b.smithi162.stdout: filter_policy: bloomfilter 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: whole_key_filtering: 1 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: verify_compression: 0 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: read_amp_bytes_per_bit: 0 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: format_version: 5 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: enable_index_compression: 1 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_align: 0 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: max_auto_readahead_size: 262144 2023-08-25T04:37:22.433 INFO:journalctl@ceph.mon.b.smithi162.stdout: prepopulate_block_cache: 0 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout: initial_auto_readahead_size: 8192 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout: num_file_reads_for_auto_readahead: 2 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.write_buffer_size: 33554432 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_write_buffer_number: 2 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression: NoCompression 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression: Disabled 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.prefix_extractor: nullptr 2023-08-25T04:37:22.434 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.num_levels: 7 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-25T04:37:22.435 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:37:22.437 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.level: 32767 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.strategy: 0 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-25T04:37:22.438 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.enabled: false 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.target_file_size_base: 67108864 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-25T04:37:22.439 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-25T04:37:22.440 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.arena_block_size: 1048576 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-25T04:37:22.441 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.disable_auto_compactions: 0 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-25T04:37:22.442 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.table_properties_collectors: 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.inplace_update_support: 0 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-25T04:37:22.443 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.bloom_locality: 0 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.max_successive_merges: 0 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.paranoid_file_checks: 0 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.force_consistency_checks: 1 2023-08-25T04:37:22.444 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.report_bg_io_stats: 0 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.ttl: 2592000 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enable_blob_files: false 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.min_blob_size: 0 2023-08-25T04:37:22.445 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_file_size: 268435456 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.blob_file_starting_level: 0 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-25T04:37:22.446 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 51361add-1efd-4f8c-a851-29b893338671 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938242249714, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938242250134, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1692938242, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "51361add-1efd-4f8c-a851-29b893338671", "db_session_id": "UM9Q61X9OZG9DCBEEU46", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: EVENT_LOG_v1 {"time_micros": 1692938242250200, "job": 1, "event": "recovery_finished"} 2023-08-25T04:37:22.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e58e5e8000 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: DB pointer 0x55e58e5d4000 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** DB Stats ** 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:37:22.448 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.449 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2023-08-25T04:37:22.450 INFO:journalctl@ceph.mon.b.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-25T04:37:22.451 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: Block cache BinnedLRUCache@0x55e58e54b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-08-25T04:37:22.452 INFO:journalctl@ceph.mon.b.smithi162.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: using public_addr v2:172.21.15.162:0/0 -> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: starting mon.b rank -1 at public addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] at bind addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 systemd[1]: Started Ceph mon.b for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:37:22.453 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(???) e0 preinit fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).mds e1 new map 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).mds e1 print_map 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: e1 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: legacy client fscid: -1 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2023-08-25T04:37:22.454 INFO:journalctl@ceph.mon.b.smithi162.stdout: No filesystems configured 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-25T04:37:22.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Added host smithi162 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:22.456 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.457 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.458 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Deploying cephadm binary to smithi183 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.459 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Added host smithi183 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:37:22.460 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:22.461 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:22.462 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:22.462 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:22.462 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:22.462 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.462 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3223805074' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.463 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Saving service mon spec with placement smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c;count:3 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:22.464 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Deploying daemon mon.c on smithi183 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.a calling monitor election 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.465 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.c calling monitor election 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-25T04:37:22.466 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: monmap e2: 2 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: fsmap 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mgrmap e13: x(active, since 45s) 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: overall HEALTH_OK 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:22.467 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: Deploying daemon mon.b on smithi162 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/2451695180' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-25T04:37:22.468 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expand map: {default=false} 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta from 'false' to 'false' 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expanded map: {default=false} 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expand map: {default=info} 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta from 'info' to 'info' 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expanded map: {default=info} 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expand map: {default=daemon} 2023-08-25T04:37:22.469 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-25T04:37:22.470 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expanded map: {default=daemon} 2023-08-25T04:37:22.470 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expand map: {default=debug} 2023-08-25T04:37:22.470 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta from 'debug' to 'debug' 2023-08-25T04:37:22.470 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:22 smithi162 ceph-mon[130941]: expand_channel_meta expanded map: {default=debug} 2023-08-25T04:37:27.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:27.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: mon.a calling monitor election 2023-08-25T04:37:27.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: mon.c calling monitor election 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: mon.b calling monitor election 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: monmap e3: 3 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: fsmap 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: mgrmap e13: x(active, since 53s) 2023-08-25T04:37:27.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: overall HEALTH_OK 2023-08-25T04:37:27.665 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.665 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.665 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.665 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:27.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: mon.a calling monitor election 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: mon.c calling monitor election 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: mon.b calling monitor election 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: monmap e3: 3 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:27.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: fsmap 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: mgrmap e13: x(active, since 53s) 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: overall HEALTH_OK 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.792+0000 7f82ada0f700 1 -- 172.21.15.183:0/3523439168 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 msgr2=0x7f8290005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:27.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.792+0000 7f82ada0f700 1 --2- 172.21.15.183:0/3523439168 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f8290005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82a80ffc40 tx=0x7f829c010e70 comp rx=0 tx=0).stop 2023-08-25T04:37:27.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.792+0000 7f82ada0f700 1 -- 172.21.15.183:0/3523439168 shutdown_connections 2023-08-25T04:37:27.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.792+0000 7f82ada0f700 1 --2- 172.21.15.183:0/3523439168 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f8290005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:27.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.792+0000 7f82ada0f700 1 -- 172.21.15.183:0/3523439168 >> 172.21.15.183:0/3523439168 conn(0x7f82a8074670 msgr2=0x7f82a8074a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:27.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.793+0000 7f82ada0f700 1 -- 172.21.15.183:0/3523439168 shutdown_connections 2023-08-25T04:37:27.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.793+0000 7f82ada0f700 1 -- 172.21.15.183:0/3523439168 wait complete. 2023-08-25T04:37:27.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.794+0000 7f82ada0f700 1 Processor -- start 2023-08-25T04:37:27.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.794+0000 7f82ada0f700 1 -- start start 2023-08-25T04:37:27.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f82a8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:27.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:27.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a819a1d0 con 0x7f82900035c0 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a81a0450 con 0x7f82a8199b70 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82ada0f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a81a05e0 con 0x7f82a8193bf0 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52096/0 (socket says 172.21.15.183:52096) 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 -- 172.21.15.183:0/1083241957 learned_addr learned my addr 172.21.15.183:0/1083241957 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 msgr2=0x7f82a81940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-08-25T04:37:27.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 msgr2=0x7f82a81940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a67fc700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a77fe700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a67fc700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 msgr2=0x7f82a81936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a67fc700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 msgr2=0x7f82a81936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a67fc700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T04:37:27.798 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f82a8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a67fc700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 msgr2=0x7f82a81940a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 msgr2=0x7f82a81936b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.795+0000 7f82a6ffd700 1 -- 172.21.15.183:0/1083241957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c0108c0 con 0x7f82900035c0 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82a6ffd700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f82a8199630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f829c020040 tx=0x7f829c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:27.799 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c0042a0 con 0x7f82900035c0 2023-08-25T04:37:27.800 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f829c01d900 con 0x7f82900035c0 2023-08-25T04:37:27.800 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c027870 con 0x7f82900035c0 2023-08-25T04:37:27.800 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a81a0860 con 0x7f82900035c0 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.796+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a81a0d30 con 0x7f82900035c0 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.797+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f829c024050 con 0x7f82900035c0 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.797+0000 7f82aca0d700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 0x7f829403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.797+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f829c04ed70 con 0x7f82900035c0 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.797+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a8061f50 con 0x7f82900035c0 2023-08-25T04:37:27.801 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.797+0000 7f82a67fc700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 0x7f829403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:27.802 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.798+0000 7f82a67fc700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 0x7f829403ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8298009850 tx=0x7f8298008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:27.802 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:27.802+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f829c02b020 con 0x7f82900035c0 2023-08-25T04:37:27.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:37:27.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: mon.a calling monitor election 2023-08-25T04:37:27.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:37:27.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: mon.c calling monitor election 2023-08-25T04:37:27.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: mon.b calling monitor election 2023-08-25T04:37:27.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:27.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:27.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-25T04:37:27.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: monmap e3: 3 mons at {a=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} 2023-08-25T04:37:27.833 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: fsmap 2023-08-25T04:37:27.833 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: osdmap e4: 0 total, 0 up, 0 in 2023-08-25T04:37:27.833 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: mgrmap e13: x(active, since 53s) 2023-08-25T04:37:27.833 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: overall HEALTH_OK 2023-08-25T04:37:27.834 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.834 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.834 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:27.834 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:28.077 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.076+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82a8105fe0 con 0x7f82900035c0 2023-08-25T04:37:28.077 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.077+0000 7f82aca0d700 1 -- 172.21.15.183:0/1083241957 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1280 (secure 0 0 0) 0x7f829c022070 con 0x7f82900035c0 2023-08-25T04:37:28.078 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-08-25T04:37:28.078 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","modified":"2023-08-25T04:37:22.281833Z","created":"2023-08-25T04:35:29.770551Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-08-25T04:37:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 msgr2=0x7f829403ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 0x7f829403ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8298009850 tx=0x7f8298008040 comp rx=0 tx=0).stop 2023-08-25T04:37:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 msgr2=0x7f82a8199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f82a8199630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f829c020040 tx=0x7f829c005790 comp rx=0 tx=0).stop 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 shutdown_connections 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8294038680 0x7f829403ab40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82a8193bf0 0x7f82a81940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82900035c0 0x7f82a8199630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 --2- 172.21.15.183:0/1083241957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a8199b70 0x7f82a81936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:28.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 >> 172.21.15.183:0/1083241957 conn(0x7f82a8074670 msgr2=0x7f82a818c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:28.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 shutdown_connections 2023-08-25T04:37:28.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:28.080+0000 7f82ada0f700 1 -- 172.21.15.183:0/1083241957 wait complete. 2023-08-25T04:37:28.082 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2023-08-25T04:37:28.490 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-08-25T04:37:28.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph config generate-minimal-conf 2023-08-25T04:37:28.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:28 smithi162 ceph-mon[130941]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:28.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:28 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/1083241957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:28.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:28 smithi183 ceph-mon[131414]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:28.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:28 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/1083241957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:28.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:28 smithi145 ceph-mon[97681]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:28.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:28 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/1083241957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-25T04:37:28.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:37:29.229 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:37:29.806 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:29.807 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.808 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:29.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:30.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 -- 172.21.15.145:0/804568401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e4072cf0 msgr2=0x7f51e40730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:30.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 --2- 172.21.15.145:0/804568401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e4072cf0 0x7f51e40730d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009910 tx=0x7f51e0009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:30.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 -- 172.21.15.145:0/804568401 shutdown_connections 2023-08-25T04:37:30.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 --2- 172.21.15.145:0/804568401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e4072cf0 0x7f51e40730d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 -- 172.21.15.145:0/804568401 >> 172.21.15.145:0/804568401 conn(0x7f51e40fbeb0 msgr2=0x7f51e40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:30.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 -- 172.21.15.145:0/804568401 shutdown_connections 2023-08-25T04:37:30.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.724+0000 7f51eb867700 1 -- 172.21.15.145:0/804568401 wait complete. 2023-08-25T04:37:30.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 Processor -- start 2023-08-25T04:37:30.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 -- start start 2023-08-25T04:37:30.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4072cf0 0x7f51e419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 0x7f51e4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e419e550 con 0x7f51e419def0 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41a47d0 con 0x7f51e4197f70 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.726+0000 7f51eb867700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41a4960 con 0x7f51e4072cf0 2023-08-25T04:37:30.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:30.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51234/0 (socket says 172.21.15.145:51234) 2023-08-25T04:37:30.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 -- 172.21.15.145:0/760933024 learned_addr learned my addr 172.21.15.145:0/760933024 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:37:30.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e9603700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4072cf0 0x7f51e419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:30.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e9e04700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 0x7f51e4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 -- 172.21.15.145:0/760933024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4072cf0 msgr2=0x7f51e419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4072cf0 0x7f51e419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 -- 172.21.15.145:0/760933024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 msgr2=0x7f51e4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 0x7f51e4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 -- 172.21.15.145:0/760933024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00095f0 con 0x7f51e419def0 2023-08-25T04:37:30.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e9e04700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 0x7f51e4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.727+0000 7f51e8e02700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b780 tx=0x7f51d400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.728+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4004230 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.728+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d4004390 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.728+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e41a4c40 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.728+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4011ad0 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.728+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41a5190 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.729+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f51d4011c30 con 0x7f51e419def0 2023-08-25T04:37:30.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.729+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e410a0e0 con 0x7f51e419def0 2023-08-25T04:37:30.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.729+0000 7f51da7fc700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 0x7f51d003ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:30.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.729+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51d404a580 con 0x7f51e419def0 2023-08-25T04:37:30.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.730+0000 7f51e9603700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 0x7f51d003ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:30.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.730+0000 7f51e9603700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 0x7f51d003ab40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009fd0 tx=0x7f51e00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:30.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.735+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f51d401ae40 con 0x7f51e419def0 2023-08-25T04:37:30.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.943+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f51e4061f50 con 0x7f51e419def0 2023-08-25T04:37:30.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.944+0000 7f51da7fc700 1 -- 172.21.15.145:0/760933024 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f51d40151c0 con 0x7f51e419def0 2023-08-25T04:37:30.947 INFO:teuthology.orchestra.run.smithi145.stdout:# minimal ceph.conf for b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:30.947 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2023-08-25T04:37:30.948 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:37:30.948 INFO:teuthology.orchestra.run.smithi145.stdout: mon_host = [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2023-08-25T04:37:30.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 msgr2=0x7f51d003ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:30.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 0x7f51d003ab40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009fd0 tx=0x7f51e00058e0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 msgr2=0x7f51e4197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:30.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b780 tx=0x7f51d400ba90 comp rx=0 tx=0).stop 2023-08-25T04:37:30.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 shutdown_connections 2023-08-25T04:37:30.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f51d0038680 0x7f51d003ab40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e4197f70 0x7f51e4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4072cf0 0x7f51e419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 --2- 172.21.15.145:0/760933024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51e419def0 0x7f51e4197a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:30.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.946+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 >> 172.21.15.145:0/760933024 conn(0x7f51e40fbeb0 msgr2=0x7f51e4107220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:30.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.947+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 shutdown_connections 2023-08-25T04:37:30.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:30.947+0000 7f51eb867700 1 -- 172.21.15.145:0/760933024 wait complete. 2023-08-25T04:37:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:31.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:31.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:31.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:31 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/760933024' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:31.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:31 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/760933024' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:31.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:31.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:31.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:31 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/760933024' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:31.515 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-08-25T04:37:31.515 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:37:31.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T04:37:31.587 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:37:31.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:31.694 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:37:31.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T04:37:31.730 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:37:31.730 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:31.801 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:37:31.802 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2023-08-25T04:37:31.837 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:37:31.838 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:37:31.909 INFO:tasks.cephadm:Adding mgr.x on smithi145 2023-08-25T04:37:31.909 INFO:tasks.cephadm:Adding mgr.y on smithi162 2023-08-25T04:37:31.910 INFO:tasks.cephadm:Adding mgr.z on smithi183 2023-08-25T04:37:31.910 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch apply mgr '3;smithi145=x;smithi162=y;smithi183=z' 2023-08-25T04:37:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T04:37:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: Reconfiguring daemon mon.a on smithi145 2023-08-25T04:37:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:32.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:32.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T04:37:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: Reconfiguring daemon mon.a on smithi145 2023-08-25T04:37:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:32.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: Reconfiguring mon.a (unknown last config time)... 2023-08-25T04:37:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: Reconfiguring daemon mon.a on smithi145 2023-08-25T04:37:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:32.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:32.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:32.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:32.502 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:37:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:33 smithi162 ceph-mon[130941]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:33 smithi162 ceph-mon[130941]: Reconfiguring mon.b (monmap changed)... 2023-08-25T04:37:33.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:33 smithi162 ceph-mon[130941]: Reconfiguring daemon mon.b on smithi162 2023-08-25T04:37:33.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:33 smithi145 ceph-mon[97681]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:33.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:33 smithi145 ceph-mon[97681]: Reconfiguring mon.b (monmap changed)... 2023-08-25T04:37:33.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:33 smithi145 ceph-mon[97681]: Reconfiguring daemon mon.b on smithi162 2023-08-25T04:37:33.418 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:33 smithi183 ceph-mon[131414]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:33.418 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:33 smithi183 ceph-mon[131414]: Reconfiguring mon.b (monmap changed)... 2023-08-25T04:37:33.419 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:33 smithi183 ceph-mon[131414]: Reconfiguring daemon mon.b on smithi162 2023-08-25T04:37:33.599 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- 172.21.15.183:0/351445851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c0ff430 msgr2=0x7f4a0c0ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 --2- 172.21.15.183:0/351445851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c0ff850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009910 tx=0x7f49fc009c20 comp rx=0 tx=0).stop 2023-08-25T04:37:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- 172.21.15.183:0/351445851 shutdown_connections 2023-08-25T04:37:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 --2- 172.21.15.183:0/351445851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c0ff850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- 172.21.15.183:0/351445851 >> 172.21.15.183:0/351445851 conn(0x7f4a0c0fa9f0 msgr2=0x7f4a0c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:33.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- 172.21.15.183:0/351445851 shutdown_connections 2023-08-25T04:37:33.601 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- 172.21.15.183:0/351445851 wait complete. 2023-08-25T04:37:33.601 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 Processor -- start 2023-08-25T04:37:33.601 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.599+0000 7f4a1244e700 1 -- start start 2023-08-25T04:37:33.602 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a0c198460 0x7f4a0c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c19ade0 0x7f4a0c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c19da10 con 0x7f4a0c19ade0 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c19db80 con 0x7f4a0c198460 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c19dd40 con 0x7f4a0c0ff430 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:33.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52102/0 (socket says 172.21.15.183:52102) 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 -- 172.21.15.183:0/3652732318 learned_addr learned my addr 172.21.15.183:0/3652732318 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a109eb700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c19ade0 0x7f4a0c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 -- 172.21.15.183:0/3652732318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a0c198460 msgr2=0x7f4a0c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a0c198460 0x7f4a0c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 -- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c19ade0 msgr2=0x7f4a0c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:33.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c19ade0 0x7f4a0c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 -- 172.21.15.183:0/3652732318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0095f0 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a0bfff700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000c00 tx=0x7f49fc004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc01d070 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc00be20 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c19dfc0 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00f460 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.600+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c19e450 con 0x7f4a0c0ff430 2023-08-25T04:37:33.605 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.601+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f49fc00f5c0 con 0x7f4a0c0ff430 2023-08-25T04:37:33.606 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.601+0000 7f4a097fa700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 0x7f49f403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:33.606 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.601+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f49fc04c400 con 0x7f4a0c0ff430 2023-08-25T04:37:33.606 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.601+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c18def0 con 0x7f4a0c0ff430 2023-08-25T04:37:33.606 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.602+0000 7f4a0b7fe700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 0x7f49f403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:33.606 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.602+0000 7f4a0b7fe700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 0x7f49f403aaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c069160 tx=0x7f4a00006dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:33.607 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.606+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f49fc027030 con 0x7f4a0c0ff430 2023-08-25T04:37:33.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.817+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi145=x;smithi162=y;smithi183=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0c02cec0 con 0x7f49f4038630 2023-08-25T04:37:33.826 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.824+0000 7f4a097fa700 1 -- 172.21.15.183:0/3652732318 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4a0c02cec0 con 0x7f49f4038630 2023-08-25T04:37:33.826 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2023-08-25T04:37:33.828 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 msgr2=0x7f49f403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 0x7f49f403aaf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c069160 tx=0x7f4a00006dd0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 msgr2=0x7f4a0c068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000c00 tx=0x7f49fc004d60 comp rx=0 tx=0).stop 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 shutdown_connections 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f49f4038630 0x7f49f403aaf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a0c19ade0 0x7f4a0c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.829 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c0ff430 0x7f4a0c068be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.830 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 --2- 172.21.15.183:0/3652732318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a0c198460 0x7f4a0c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:33.830 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 >> 172.21.15.183:0/3652732318 conn(0x7f4a0c0fa9f0 msgr2=0x7f4a0c0fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:33.830 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 shutdown_connections 2023-08-25T04:37:33.830 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:37:33.827+0000 7f4a1244e700 1 -- 172.21.15.183:0/3652732318 wait complete. 2023-08-25T04:37:34.247 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.247 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.247 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:34.248 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:34.248 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:34.248 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:34.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:34.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-25T04:37:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-25T04:37:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:34.649 DEBUG:teuthology.orchestra.run.smithi162:mgr.y> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.y.service 2023-08-25T04:37:34.652 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.z.service 2023-08-25T04:37:34.654 INFO:tasks.cephadm:Deploying OSDs... 2023-08-25T04:37:34.654 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:37:34.655 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T04:37:34.682 INFO:journalctl@ceph.mgr.z.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:37:34.687 INFO:journalctl@ceph.mgr.y.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:37:34.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T04:37:34.703 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2023-08-25T04:37:34.789 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T04:37:34.789 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:34.789 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-08-25 04:36:45.225786612 +0000 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-08-25 04:30:53.035243596 +0000 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-08-25 04:30:53.035243596 +0000 2023-08-25T04:37:34.790 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2023-08-25 04:30:53.007244173 +0000 2023-08-25T04:37:34.791 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T04:37:34.857 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-08-25T04:37:34.858 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-08-25T04:37:34.858 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000306587 s, 1.7 MB/s 2023-08-25T04:37:34.860 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T04:37:34.905 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2023-08-25T04:37:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T04:37:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:34.989 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-08-25T04:37:34.989 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:34.989 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:34.990 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-08-25 04:36:45.225786612 +0000 2023-08-25T04:37:34.990 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-08-25 04:30:53.847226853 +0000 2023-08-25T04:37:34.990 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-08-25 04:30:53.847226853 +0000 2023-08-25T04:37:34.990 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2023-08-25 04:30:53.820227410 +0000 2023-08-25T04:37:34.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T04:37:35.096 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-08-25T04:37:35.097 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-08-25T04:37:35.097 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000311384 s, 1.6 MB/s 2023-08-25T04:37:35.099 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T04:37:35.145 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:35.229 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-08-25 04:36:45.225786612 +0000 2023-08-25T04:37:35.230 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-08-25 04:30:54.683209616 +0000 2023-08-25T04:37:35.230 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-08-25 04:30:54.683209616 +0000 2023-08-25T04:37:35.230 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2023-08-25 04:30:54.656210172 +0000 2023-08-25T04:37:35.230 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T04:37:35.286 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: Reconfiguring mon.c (monmap changed)... 2023-08-25T04:37:35.286 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: Reconfiguring daemon mon.c on smithi183 2023-08-25T04:37:35.286 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi145=x;smithi162=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: Saving service mgr spec with placement smithi145=x;smithi162=y;smithi183=z;count:3 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:35.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.288 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:35.288 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:35.288 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:35.288 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: Reconfiguring mon.c (monmap changed)... 2023-08-25T04:37:35.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: Reconfiguring daemon mon.c on smithi183 2023-08-25T04:37:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi145=x;smithi162=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: Saving service mgr spec with placement smithi145=x;smithi162=y;smithi183=z;count:3 2023-08-25T04:37:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:35.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:35.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:35.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:35.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:35.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.333 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-08-25T04:37:35.334 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-08-25T04:37:35.334 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.00028479 s, 1.8 MB/s 2023-08-25T04:37:35.336 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T04:37:35.381 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2023-08-25T04:37:35.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: Reconfiguring mon.c (monmap changed)... 2023-08-25T04:37:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: Reconfiguring daemon mon.c on smithi183 2023-08-25T04:37:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi145=x;smithi162=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: Saving service mgr spec with placement smithi145=x;smithi162=y;smithi183=z;count:3 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:35.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:35.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:35.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:35.465 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-08-25 04:36:45.225786612 +0000 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-08-25 04:30:55.597190770 +0000 2023-08-25T04:37:35.466 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-08-25 04:30:55.597190770 +0000 2023-08-25T04:37:35.467 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2023-08-25 04:30:55.570191327 +0000 2023-08-25T04:37:35.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T04:37:35.571 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-08-25T04:37:35.571 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-08-25T04:37:35.571 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000329585 s, 1.6 MB/s 2023-08-25T04:37:35.573 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T04:37:35.618 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-08-25T04:37:35.618 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T04:37:35.640 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T04:37:35.640 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2023-08-25T04:37:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T04:37:35.700 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:35.700 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 385451 Links: 1 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-08-25 04:37:21.718904255 +0000 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-08-25 04:27:48.338670176 +0000 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-08-25 04:27:48.338670176 +0000 2023-08-25T04:37:35.701 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2023-08-25T04:37:35.702 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T04:37:35.770 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-08-25T04:37:35.771 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-08-25T04:37:35.771 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000242585 s, 2.1 MB/s 2023-08-25T04:37:35.772 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T04:37:35.832 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2023-08-25T04:37:35.892 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T04:37:35.893 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:35.893 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 389310 Links: 1 2023-08-25T04:37:35.894 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:35.894 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:35.894 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-08-25 04:37:21.718904255 +0000 2023-08-25T04:37:35.894 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-08-25 04:27:48.907658262 +0000 2023-08-25T04:37:35.894 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-08-25 04:27:48.907658262 +0000 2023-08-25T04:37:35.895 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2023-08-25T04:37:35.895 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T04:37:35.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:35 smithi183 systemd[1]: Starting Ceph mgr.z for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:37:35.960 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-08-25T04:37:35.960 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-08-25T04:37:35.961 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000226868 s, 2.3 MB/s 2023-08-25T04:37:35.962 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T04:37:36.020 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2023-08-25T04:37:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T04:37:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 385640 Links: 1 2023-08-25T04:37:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-08-25 04:37:21.719904233 +0000 2023-08-25T04:37:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-08-25 04:27:49.450646899 +0000 2023-08-25T04:37:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-08-25 04:27:49.450646899 +0000 2023-08-25T04:37:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2023-08-25T04:37:36.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T04:37:36.145 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-08-25T04:37:36.145 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-08-25T04:37:36.146 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000274565 s, 1.9 MB/s 2023-08-25T04:37:36.147 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T04:37:36.205 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2023-08-25T04:37:36.263 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T04:37:36.263 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.263 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 385715 Links: 1 2023-08-25T04:37:36.264 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.264 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.264 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-08-25 04:37:21.719904233 +0000 2023-08-25T04:37:36.264 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-08-25 04:27:49.996635472 +0000 2023-08-25T04:37:36.265 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-08-25 04:27:49.996635472 +0000 2023-08-25T04:37:36.265 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2023-08-25T04:37:36.265 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T04:37:36.328 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-08-25T04:37:36.328 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-08-25T04:37:36.329 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000205279 s, 2.5 MB/s 2023-08-25T04:37:36.329 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T04:37:36.387 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-08-25T04:37:36.387 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T04:37:36.405 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T04:37:36.406 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2023-08-25T04:37:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:36 smithi162 ceph-mon[130941]: Deploying daemon mgr.z on smithi183 2023-08-25T04:37:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-mon[131414]: Deploying daemon mgr.z on smithi183 2023-08-25T04:37:36.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 podman[133264]: 2023-08-25T04:37:36.440 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 386941 Links: 1 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-08-25 04:37:35.820945064 +0000 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-08-25 04:27:31.966030175 +0000 2023-08-25T04:37:36.441 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-08-25 04:27:31.966030175 +0000 2023-08-25T04:37:36.442 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-08-25T04:37:36.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T04:37:36.511 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-08-25T04:37:36.511 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-08-25T04:37:36.511 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000234628 s, 2.2 MB/s 2023-08-25T04:37:36.512 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T04:37:36.569 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2023-08-25T04:37:36.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:36 smithi145 ceph-mon[97681]: Deploying daemon mgr.z on smithi183 2023-08-25T04:37:36.626 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T04:37:36.626 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.626 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 389110 Links: 1 2023-08-25T04:37:36.626 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.626 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.627 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-08-25 04:37:35.820945064 +0000 2023-08-25T04:37:36.627 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-08-25 04:27:32.530015268 +0000 2023-08-25T04:37:36.627 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-08-25 04:27:32.530015268 +0000 2023-08-25T04:37:36.627 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-08-25T04:37:36.627 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T04:37:36.691 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-mgr[133323]: -- 172.21.15.183:0/595040303 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b562371e0 con 0x555b5620c800 2023-08-25T04:37:36.691 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 bash[133264]: 5838764ef720feb727075c9861efef3cddd2297e6802683ddd178f648b20fc90 2023-08-25T04:37:36.691 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 systemd[1]: Started Ceph mgr.z for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:37:36.691 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:36.561+0000 7f2c8016b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:37:36.692 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:36.643+0000 7f2c8016b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:37:36.693 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-08-25T04:37:36.693 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-08-25T04:37:36.693 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000218185 s, 2.3 MB/s 2023-08-25T04:37:36.694 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T04:37:36.752 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2023-08-25T04:37:36.809 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T04:37:36.809 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.809 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 391273 Links: 1 2023-08-25T04:37:36.809 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.809 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.810 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-08-25 04:37:35.820945064 +0000 2023-08-25T04:37:36.810 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-08-25 04:27:33.079000757 +0000 2023-08-25T04:37:36.811 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-08-25 04:27:33.079000757 +0000 2023-08-25T04:37:36.811 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-08-25T04:37:36.811 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T04:37:36.875 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-08-25T04:37:36.875 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-08-25T04:37:36.875 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000209089 s, 2.4 MB/s 2023-08-25T04:37:36.877 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T04:37:36.935 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2023-08-25T04:37:36.992 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 384653 Links: 1 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-08-25 04:37:35.820945064 +0000 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-08-25 04:27:33.626986274 +0000 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-08-25 04:27:33.626986274 +0000 2023-08-25T04:37:36.993 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-08-25T04:37:36.994 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T04:37:37.021 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:36.740+0000 7f2c8016b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:37:37.021 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:36.838+0000 7f2c8016b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:37:37.046 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-08-25T04:37:37.046 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-08-25T04:37:37.047 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000260013 s, 2.0 MB/s 2023-08-25T04:37:37.047 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T04:37:37.106 INFO:tasks.cephadm:Deploying osd.0 on smithi145 with /dev/vg_nvme/lv_4... 2023-08-25T04:37:37.106 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T04:37:37.395 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:37.019+0000 7f2c8016b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:37.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:37.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:37.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:37.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-mon[131414]: Deploying daemon mgr.y on smithi162 2023-08-25T04:37:37.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:37.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:37.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:37.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:37 smithi162 ceph-mon[130941]: Deploying daemon mgr.y on smithi162 2023-08-25T04:37:37.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:37.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:37.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:37.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-25T04:37:37.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:37.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:37.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:37 smithi145 ceph-mon[97681]: Deploying daemon mgr.y on smithi162 2023-08-25T04:37:37.674 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:37.393+0000 7f2c8016b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:37:37.674 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:37.524+0000 7f2c8016b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:37:37.674 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:37.611+0000 7f2c8016b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:37:37.832 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:37:38.090 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:37 smithi162 systemd[1]: Starting Ceph mgr.y for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:37:38.091 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:37.705+0000 7f2c8016b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:37:38.364 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:38 smithi145 ceph-mon[97681]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:38.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-mon[130941]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:38.366 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 podman[132376]: 2023-08-25T04:37:38.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:38 smithi183 ceph-mon[131414]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:38.425 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:38.123+0000 7f2c8016b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:37:38.661 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 bash[132376]: 5324331668e7ee100a42419d3434f83136b000811eaa56f3e17154f7fb9a76ed 2023-08-25T04:37:38.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-mgr[132416]: -- 172.21.15.162:0/429391698 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a577b91e0 con 0x558a5778f000 2023-08-25T04:37:38.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 systemd[1]: Started Ceph mgr.y for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:37:38.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:38.501+0000 7f7abc972200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:37:38.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:38.586+0000 7f7abc972200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:37:38.954 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:38.675+0000 7f7abc972200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:37:38.954 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:38.773+0000 7f7abc972200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:37:39.172 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:38.793+0000 7f2c8016b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:37:39.172 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:38.884+0000 7f2c8016b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:37:39.353 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:38.953+0000 7f7abc972200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:37:39.353 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:39.352+0000 7f7abc972200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:37:39.661 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:39.488+0000 7f7abc972200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:37:39.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:39.578+0000 7f7abc972200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:37:39.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:39.720 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T04:37:39.721 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T04:37:39.721 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2023-08-25T04:37:39.721 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2023-08-25T04:37:39.721 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0390816 s, 268 MB/s 2023-08-25T04:37:39.721 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2023-08-25T04:37:39.800 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:39.700+0000 7f2c8016b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:37:39.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.802 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.802 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:39.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:39.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:40.140 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:39.687+0000 7f7abc972200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:37:40.172 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:39.799+0000 7f2c8016b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:37:40.172 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:39.992+0000 7f2c8016b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:37:40.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2023-08-25T04:37:40.412 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:40.139+0000 7f7abc972200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:37:40.671 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:40.289+0000 7f2c8016b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:37:40.671 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:40.517+0000 7f2c8016b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:37:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:40.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:40.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:40.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:40.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.830 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:40.830 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:40.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:40.832 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:40.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:40.669+0000 7f2c8016b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:37:40.952 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:37:41.161 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:40.829+0000 7f7abc972200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:37:41.162 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:40.911+0000 7f7abc972200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:37:41.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:40.931+0000 7f2c8016b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:37:41.793 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:41 smithi145 ceph-mon[97681]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T04:37:41.794 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:41.794 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:41.794 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:41.794 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:41 smithi145 ceph-mon[97681]: Reconfiguring daemon mgr.x on smithi145 2023-08-25T04:37:41.911 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:41.681+0000 7f7abc972200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:37:41.912 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:41.771+0000 7f7abc972200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:37:41.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-mon[130941]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T04:37:41.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:41.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:41.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:41.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-mon[130941]: Reconfiguring daemon mgr.x on smithi145 2023-08-25T04:37:41.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:41.623+0000 7f2c8016b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:37:41.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:41.753+0000 7f2c8016b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:37:41.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-mon[131414]: Reconfiguring mgr.x (unknown last config time)... 2023-08-25T04:37:41.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-25T04:37:41.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-25T04:37:41.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:41.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-mon[131414]: Reconfiguring daemon mgr.x on smithi145 2023-08-25T04:37:42.219 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:41.943+0000 7f7abc972200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:37:42.270 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:41.943+0000 7f2c8016b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:37:42.271 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:42.054+0000 7f2c8016b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:37:42.536 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:42.269+0000 7f2c8016b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:37:42.537 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:42.218+0000 7f7abc972200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:37:42.537 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:42.445+0000 7f7abc972200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:37:42.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 -- 172.21.15.145:0/3713637161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730108880 msgr2=0x7f473010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:42.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 --2- 172.21.15.145:0/3713637161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730108880 0x7f473010ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4728009f90 tx=0x7f472800b330 comp rx=0 tx=0).stop 2023-08-25T04:37:42.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 -- 172.21.15.145:0/3713637161 shutdown_connections 2023-08-25T04:37:42.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 --2- 172.21.15.145:0/3713637161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730108880 0x7f473010ad10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:42.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 --2- 172.21.15.145:0/3713637161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730074950 0x7f4730072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:42.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 --2- 172.21.15.145:0/3713637161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f4730073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:42.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.699+0000 7f473494b700 1 -- 172.21.15.145:0/3713637161 >> 172.21.15.145:0/3713637161 conn(0x7f47300fbf10 msgr2=0x7f47300fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:37:42.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.700+0000 7f473494b700 1 -- 172.21.15.145:0/3713637161 shutdown_connections 2023-08-25T04:37:42.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.700+0000 7f473494b700 1 -- 172.21.15.145:0/3713637161 wait complete. 2023-08-25T04:37:42.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 Processor -- start 2023-08-25T04:37:42.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 -- start start 2023-08-25T04:37:42.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:42.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730074950 0x7f4730198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:42.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730108880 0x7f473019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:42.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a0580 con 0x7f4730074950 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a06c0 con 0x7f4730073380 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f473494b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301a0850 con 0x7f4730108880 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f472e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f472e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:36520/0 (socket says 172.21.15.145:36520) 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.701+0000 7f472e59c700 1 -- 172.21.15.145:0/994646422 learned_addr learned my addr 172.21.15.145:0/994646422 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:37:42.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472dd9b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730074950 0x7f4730198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472ed9d700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730108880 0x7f473019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 -- 172.21.15.145:0/994646422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730108880 msgr2=0x7f473019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730108880 0x7f473019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 -- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730074950 msgr2=0x7f4730198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730074950 0x7f4730198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 -- 172.21.15.145:0/994646422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47240095f0 con 0x7f4730073380 2023-08-25T04:37:42.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f472e59c700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f472000e970 tx=0x7f472000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:42.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720009790 con 0x7f4730073380 2023-08-25T04:37:42.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472000c7e0 con 0x7f4730073380 2023-08-25T04:37:42.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728009c40 con 0x7f4730073380 2023-08-25T04:37:42.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.702+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200054e0 con 0x7f4730073380 2023-08-25T04:37:42.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.703+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a0ea0 con 0x7f4730073380 2023-08-25T04:37:42.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.704+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 45192+0+0 (secure 0 0 0) 0x7f4720021440 con 0x7f4730073380 2023-08-25T04:37:42.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.704+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4730106f30 con 0x7f4730073380 2023-08-25T04:37:42.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.704+0000 7f471f7fe700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 0x7f471803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:37:42.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.704+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4720014070 con 0x7f4730073380 2023-08-25T04:37:42.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.704+0000 7f472dd9b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 0x7f471803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:37:42.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.705+0000 7f472dd9b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 0x7f471803a8a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4730074680 tx=0x7f4724009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:37:42.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.709+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4720004a30 con 0x7f4730073380 2023-08-25T04:37:42.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:42 smithi145 ceph-mon[97681]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:42.863 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-mon[130941]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:42.863 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:42.863 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:42.864 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:42.593+0000 7f7abc972200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:37:42.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:42.920+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f473002cea0 con 0x7f47180383e0 2023-08-25T04:37:42.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:42 smithi183 ceph-mon[131414]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:42.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:42.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:43.161 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:42.862+0000 7f7abc972200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:37:43.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:37:43.126+0000 7f2c8016b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:37:43.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:43.547+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 14) v1 ==== 89904+0+0 (secure 0 0 0) 0x7f472000fca0 con 0x7f4730073380 2023-08-25T04:37:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: Standby manager daemon z started 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:43.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:37:43.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:43 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:43.883 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:43.561+0000 7f7abc972200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:37:43.883 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:43.691+0000 7f7abc972200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:37:43.883 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:43.883 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:37:43.884 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: Standby manager daemon z started 2023-08-25T04:37:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:37:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:37:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:43.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:37:43.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:43.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: Standby manager daemon z started 2023-08-25T04:37:43.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:37:43.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:43.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:37:43.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:43 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/2864127429' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:44.161 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:43.882+0000 7f7abc972200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:37:44.162 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:43.993+0000 7f7abc972200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:37:44.550 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:44.209+0000 7f7abc972200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:37:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:44 smithi145 ceph-mon[97681]: mgrmap e14: x(active, since 69s), standbys: z 2023-08-25T04:37:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:37:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:44 smithi145 ceph-mon[97681]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:44.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:44 smithi162 ceph-mon[130941]: mgrmap e14: x(active, since 69s), standbys: z 2023-08-25T04:37:44.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:44 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:37:44.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:44 smithi162 ceph-mon[130941]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:44.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:44 smithi183 ceph-mon[131414]: mgrmap e14: x(active, since 69s), standbys: z 2023-08-25T04:37:44.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:44 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:37:44.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:44 smithi183 ceph-mon[131414]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:45.411 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:37:45.067+0000 7f7abc972200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:37:45.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:37:45.561+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mon.2 v2:172.21.15.162:3300/0 8 ==== mgrmap(e 15) v1 ==== 134616+0+0 (secure 0 0 0) 0x7f4720014230 con 0x7f4730073380 2023-08-25T04:37:45.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:45 smithi145 ceph-mon[97681]: Standby manager daemon y started 2023-08-25T04:37:45.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:45 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:37:45.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:45 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:45.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:45 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:37:45.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:45 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:45.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-mon[130941]: Standby manager daemon y started 2023-08-25T04:37:45.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:37:45.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:45.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:37:45.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:45 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:45.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:45 smithi183 ceph-mon[131414]: Standby manager daemon y started 2023-08-25T04:37:45.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:45 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:37:45.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:45 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:37:45.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:45 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:37:45.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:45 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/3873956913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:37:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-08-25T04:37:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:37:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]: dispatch 2023-08-25T04:37:46.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]': finished 2023-08-25T04:37:46.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T04:37:46.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:37:46.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-08-25T04:37:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:37:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]: dispatch 2023-08-25T04:37:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]': finished 2023-08-25T04:37:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T04:37:46.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:37:46.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]: dispatch 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/819094910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b0e0c3-39e3-4870-b353-a2e7c3811368"}]': finished 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: osdmap e5: 1 total, 0 up, 1 in 2023-08-25T04:37:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:37:47.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:47 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2662139284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:37:47.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:47 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2662139284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:37:47.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:47 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2662139284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:37:48.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:48 smithi145 ceph-mon[97681]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:48.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:48 smithi162 ceph-mon[130941]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:48.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:48 smithi183 ceph-mon[131414]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:50.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:50 smithi145 ceph-mon[97681]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:50.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:50 smithi162 ceph-mon[130941]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:50.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:50 smithi183 ceph-mon[131414]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:52.945 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:52 smithi145 ceph-mon[97681]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:53.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:52 smithi162 ceph-mon[130941]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:53.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:52 smithi183 ceph-mon[131414]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:53 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T04:37:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:53 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:53 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T04:37:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:53 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:53 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-25T04:37:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:53 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:37:54.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:54 smithi145 ceph-mon[97681]: Deploying daemon osd.0 on smithi145 2023-08-25T04:37:54.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:54 smithi145 ceph-mon[97681]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:55.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:54 smithi162 ceph-mon[130941]: Deploying daemon osd.0 on smithi145 2023-08-25T04:37:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:54 smithi162 ceph-mon[130941]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:55.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:54 smithi183 ceph-mon[131414]: Deploying daemon osd.0 on smithi145 2023-08-25T04:37:55.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:54 smithi183 ceph-mon[131414]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:56 smithi145 ceph-mon[97681]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:57.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:56 smithi162 ceph-mon[130941]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:57.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:56 smithi183 ceph-mon[131414]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:59.033 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:37:58 smithi145 ceph-mon[97681]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:59.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:37:58 smithi162 ceph-mon[130941]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:37:59.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:37:58 smithi183 ceph-mon[131414]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:00.641 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.642 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.642 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:00.642 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:00 smithi145 ceph-mon[97681]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:00.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:00 smithi162 ceph-mon[130941]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:00.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:00.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:00.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:00 smithi183 ceph-mon[131414]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:02.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.778+0000 7f471f7fe700 1 -- 172.21.15.145:0/994646422 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f473002cea0 con 0x7f47180383e0 2023-08-25T04:38:02.780 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 0 on host 'smithi145' 2023-08-25T04:38:02.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 msgr2=0x7f471803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:02.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 0x7f471803a8a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4730074680 tx=0x7f4724009310 comp rx=0 tx=0).stop 2023-08-25T04:38:02.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 msgr2=0x7f47301989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:02.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f472000e970 tx=0x7f472000ec80 comp rx=0 tx=0).stop 2023-08-25T04:38:02.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 shutdown_connections 2023-08-25T04:38:02.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f47180383e0 0x7f471803a8a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:02.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4730108880 0x7f473019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:02.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4730073380 0x7f47301989f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:02.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 --2- 172.21.15.145:0/994646422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4730074950 0x7f4730198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:02.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 >> 172.21.15.145:0/994646422 conn(0x7f47300fbf10 msgr2=0x7f47300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:02.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 shutdown_connections 2023-08-25T04:38:02.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:02.782+0000 7f473494b700 1 -- 172.21.15.145:0/994646422 wait complete. 2023-08-25T04:38:02.909 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:02.910 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.910 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.910 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:02.911 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:02.911 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:02.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:03.174 DEBUG:teuthology.orchestra.run.smithi145:osd.0> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.0.service 2023-08-25T04:38:03.179 INFO:tasks.cephadm:Deploying osd.1 on smithi162 with /dev/vg_nvme/lv_4... 2023-08-25T04:38:03.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T04:38:03.250 INFO:journalctl@ceph.osd.0.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:38:03.577 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T04:38:03.344+0000 7f872c799640 -1 osd.0 0 log_to_monitors true 2023-08-25T04:38:03.743 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:38:03.988 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:03.989 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:03.989 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:03.989 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:03 smithi162 ceph-mon[130941]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:03.990 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:03 smithi162 ceph-mon[130941]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:04.008 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:04.008 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:04.008 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:04.009 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-mon[97681]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:04.009 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:03 smithi145 ceph-mon[97681]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:04.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:04.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:04.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:03 smithi183 ceph-mon[131414]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:04.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:03 smithi183 ceph-mon[131414]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-25T04:38:04.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T04:38:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T04:38:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:04 smithi162 ceph-mon[130941]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:05.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T04:38:04.750+0000 7f8721e10700 -1 osd.0 0 waiting for initial osdmap 2023-08-25T04:38:05.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T04:38:04.754+0000 7f871bc01700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:05.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:04 smithi145 ceph-mon[97681]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:05.130 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T04:38:05.130 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T04:38:05.130 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2023-08-25T04:38:05.131 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2023-08-25T04:38:05.131 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110422 s, 950 MB/s 2023-08-25T04:38:05.131 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2023-08-25T04:38:05.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:05.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-25T04:38:05.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: osdmap e6: 1 total, 0 up, 1 in 2023-08-25T04:38:05.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:05.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: from='osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:05.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:04 smithi183 ceph-mon[131414]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:38:05.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2023-08-25T04:38:06.046 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:38:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:05 smithi145 ceph-mon[97681]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:38:06.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:05 smithi145 ceph-mon[97681]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T04:38:06.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:05 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:05 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:05 smithi162 ceph-mon[130941]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:38:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:05 smithi162 ceph-mon[130941]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T04:38:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:05 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:05 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:05 smithi183 ceph-mon[131414]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:38:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:05 smithi183 ceph-mon[131414]: osdmap e7: 1 total, 0 up, 1 in 2023-08-25T04:38:06.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:05 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:05 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:38:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:38:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] boot 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi145 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:06.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 17267M 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:06.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:38:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:38:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] boot 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi145 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:07.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 17267M 2023-08-25T04:38:07.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:07.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:07.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:38:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:38:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-25T04:38:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: osd.0 [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] boot 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: osdmap e8: 1 total, 1 up, 1 in 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi145 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:07.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 17267M 2023-08-25T04:38:07.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:07.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:07.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:07.240 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.237+0000 7fd3566ee700 1 -- 172.21.15.162:0/3782175951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 msgr2=0x7fd350100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:07.240 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3782175951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350100f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3440099c0 tx=0x7fd344009cd0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.240 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 -- 172.21.15.162:0/3782175951 shutdown_connections 2023-08-25T04:38:07.240 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3782175951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd350107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.240 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3782175951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 0x7fd3500ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3782175951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350100f10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 -- 172.21.15.162:0/3782175951 >> 172.21.15.162:0/3782175951 conn(0x7fd3500fae30 msgr2=0x7fd3500fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 -- 172.21.15.162:0/3782175951 shutdown_connections 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 -- 172.21.15.162:0/3782175951 wait complete. 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 Processor -- start 2023-08-25T04:38:07.241 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.238+0000 7fd3566ee700 1 -- start start 2023-08-25T04:38:07.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 0x7fd350198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:07.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:07.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd35019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:07.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a03e0 con 0x7fd3500ff8b0 2023-08-25T04:38:07.243 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a0520 con 0x7fd350100ab0 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a06b0 con 0x7fd350101450 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:32954/0 (socket says 172.21.15.162:32954) 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 -- 172.21.15.162:0/3740922230 learned_addr learned my addr 172.21.15.162:0/3740922230 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34ffff700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 0x7fd350198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:07.244 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd354c8b700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd35019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 -- 172.21.15.162:0/3740922230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 msgr2=0x7fd35019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd35019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 -- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 msgr2=0x7fd350198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 0x7fd350198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 -- 172.21.15.162:0/3740922230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440095f0 con 0x7fd350100ab0 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd354c8b700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd35019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:38:07.245 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34f7fe700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd344009990 tx=0x7fd344005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:07.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34401d070 con 0x7fd350100ab0 2023-08-25T04:38:07.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd344022470 con 0x7fd350100ab0 2023-08-25T04:38:07.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344004230 con 0x7fd350100ab0 2023-08-25T04:38:07.246 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3501a0930 con 0x7fd350100ab0 2023-08-25T04:38:07.247 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.239+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3501a0dd0 con 0x7fd350100ab0 2023-08-25T04:38:07.247 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.240+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35010b0b0 con 0x7fd350100ab0 2023-08-25T04:38:07.247 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.241+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 134616+0+0 (secure 0 0 0) 0x7fd3440225e0 con 0x7fd350100ab0 2023-08-25T04:38:07.247 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.242+0000 7fd34d7fa700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 0x7fd3380a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:07.248 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.242+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd3440ca910 con 0x7fd350100ab0 2023-08-25T04:38:07.248 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.242+0000 7fd34ffff700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 0x7fd3380a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:07.248 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.243+0000 7fd34ffff700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 0x7fd3380a2400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd340005950 tx=0x7fd3400058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:07.248 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.246+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd3440961a0 con 0x7fd350100ab0 2023-08-25T04:38:07.443 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:07.442+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd350068bb0 con 0x7fd33809ff40 2023-08-25T04:38:08.263 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:08 smithi162 ceph-mon[130941]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T04:38:08.264 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:08.264 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:08.264 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:08 smithi145 ceph-mon[97681]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T04:38:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:08 smithi183 ceph-mon[131414]: osdmap e9: 1 total, 1 up, 1 in 2023-08-25T04:38:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:08.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:08.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:09 smithi145 ceph-mon[97681]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:09 smithi145 ceph-mon[97681]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:09.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:09 smithi162 ceph-mon[130941]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:09.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:09 smithi162 ceph-mon[130941]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:09.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:09 smithi183 ceph-mon[131414]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:09.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:09 smithi183 ceph-mon[131414]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/2503283025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]': finished 2023-08-25T04:38:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T04:38:10.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:10.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/519013435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:10.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:10.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/2503283025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]': finished 2023-08-25T04:38:10.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T04:38:10.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:10.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/519013435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:10.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/2503283025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]: dispatch 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825a87ec-1ec6-46eb-9e65-2aed8e37f8ae"}]': finished 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: osdmap e10: 2 total, 1 up, 2 in 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/519013435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:13.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:12 smithi145 ceph-mon[97681]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:13.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:12 smithi162 ceph-mon[130941]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:13.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:12 smithi183 ceph-mon[131414]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:14.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:14 smithi162 ceph-mon[130941]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:14 smithi145 ceph-mon[97681]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:14 smithi183 ceph-mon[131414]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:16.991 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:16 smithi162 ceph-mon[130941]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:16.991 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T04:38:16.991 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:17.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:16 smithi145 ceph-mon[97681]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T04:38:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:16 smithi183 ceph-mon[131414]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-25T04:38:17.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:17 smithi145 ceph-mon[97681]: Deploying daemon osd.1 on smithi162 2023-08-25T04:38:18.096 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:17 smithi162 ceph-mon[130941]: Deploying daemon osd.1 on smithi162 2023-08-25T04:38:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:17 smithi183 ceph-mon[131414]: Deploying daemon osd.1 on smithi162 2023-08-25T04:38:19.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:18 smithi145 ceph-mon[97681]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:18 smithi162 ceph-mon[130941]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:18 smithi183 ceph-mon[131414]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:20.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:20 smithi145 ceph-mon[97681]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:20.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:20 smithi162 ceph-mon[130941]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:20.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:20.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:20 smithi183 ceph-mon[131414]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:22.761 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 1 on host 'smithi162' 2023-08-25T04:38:22.762 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.760+0000 7fd34d7fa700 1 -- 172.21.15.162:0/3740922230 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd350068bb0 con 0x7fd33809ff40 2023-08-25T04:38:22.763 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.762+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 msgr2=0x7fd3380a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:22.763 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.762+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 0x7fd3380a2400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd340005950 tx=0x7fd3400058e0 comp rx=0 tx=0).stop 2023-08-25T04:38:22.763 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.762+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 msgr2=0x7fd350198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.762+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd344009990 tx=0x7fd344005e80 comp rx=0 tx=0).stop 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 shutdown_connections 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd33809ff40 0x7fd3380a2400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd350101450 0x7fd35019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3500ff8b0 0x7fd350198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 --2- 172.21.15.162:0/3740922230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd350100ab0 0x7fd350198d90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:22.764 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 >> 172.21.15.162:0/3740922230 conn(0x7fd3500fae30 msgr2=0x7fd3500684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:22.765 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 shutdown_connections 2023-08-25T04:38:22.765 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:38:22.763+0000 7fd3566ee700 1 -- 172.21.15.162:0/3740922230 wait complete. 2023-08-25T04:38:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:23.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:23.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.112 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:23.113 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.113 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.113 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:23.113 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:23.113 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.137 DEBUG:teuthology.orchestra.run.smithi162:osd.1> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.1.service 2023-08-25T04:38:23.139 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2023-08-25T04:38:23.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-25T04:38:23.170 INFO:journalctl@ceph.osd.1.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:38:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:23.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:23.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.701 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:38:23.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:24.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:24.084 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:24.085 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:24.355 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:24 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T04:38:24.246+0000 7f017b363640 -1 osd.1 0 log_to_monitors true 2023-08-25T04:38:25.048 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:24 smithi162 ceph-mon[130941]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:25.048 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:24 smithi162 ceph-mon[130941]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.048 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:24 smithi162 ceph-mon[130941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:24 smithi145 ceph-mon[97681]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:24 smithi145 ceph-mon[97681]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:24 smithi145 ceph-mon[97681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:24 smithi183 ceph-mon[131414]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:24 smithi183 ceph-mon[131414]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:24 smithi183 ceph-mon[131414]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135479 s, 774 MB/s 2023-08-25T04:38:25.182 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2023-08-25T04:38:25.561 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2023-08-25T04:38:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T04:38:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T04:38:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:26.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:26.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:26.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:25 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.152 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:38:26.161 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T04:38:25.742+0000 7f01709da700 -1 osd.1 0 waiting for initial osdmap 2023-08-25T04:38:26.162 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T04:38:25.746+0000 7f016a7cb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:38:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T04:38:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T04:38:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:26.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:26.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:25 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-25T04:38:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: osdmap e11: 2 total, 1 up, 2 in 2023-08-25T04:38:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:25 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:26.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi162 2023-08-25T04:38:26.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 17168M 2023-08-25T04:38:26.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:26.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:38:26.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T04:38:26.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:26.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:27.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi162 2023-08-25T04:38:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 17168M 2023-08-25T04:38:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:38:27.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T04:38:27.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:27.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi162 2023-08-25T04:38:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 17168M 2023-08-25T04:38:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:38:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: osdmap e12: 2 total, 1 up, 2 in 2023-08-25T04:38:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:27.339 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 -- 172.21.15.183:0/3349635419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 msgr2=0x7f5420107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:27.339 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 --2- 172.21.15.183:0/3349635419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f5420107d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f541800b0a0 tx=0x7f541800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.339 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 -- 172.21.15.183:0/3349635419 shutdown_connections 2023-08-25T04:38:27.339 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 --2- 172.21.15.183:0/3349635419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f5420107d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 --2- 172.21.15.183:0/3349635419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54200ff8b0 0x7f54200ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 --2- 172.21.15.183:0/3349635419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5420100ab0 0x7f5420100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 -- 172.21.15.183:0/3349635419 >> 172.21.15.183:0/3349635419 conn(0x7f54200fae30 msgr2=0x7f54200fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 -- 172.21.15.183:0/3349635419 shutdown_connections 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.337+0000 7f542661c700 1 -- 172.21.15.183:0/3349635419 wait complete. 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 Processor -- start 2023-08-25T04:38:27.340 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 -- start start 2023-08-25T04:38:27.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 0x7f5420196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:27.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5420100ab0 0x7f5420196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:27.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e3a0 con 0x7f5420100ab0 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e4e0 con 0x7f54200ff8b0 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f542661c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e670 con 0x7f5420101450 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52136/0 (socket says 172.21.15.183:52136) 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 -- 172.21.15.183:0/1546516522 learned_addr learned my addr 172.21.15.183:0/1546516522 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f541ffff700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 0x7f5420196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:27.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f541f7fe700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5420100ab0 0x7f5420196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 -- 172.21.15.183:0/1546516522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 msgr2=0x7f5420196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 0x7f5420196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 -- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5420100ab0 msgr2=0x7f5420196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5420100ab0 0x7f5420196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f5424bb9700 1 -- 172.21.15.183:0/1546516522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009d00 con 0x7f5420101450 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.338+0000 7f541ffff700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 0x7f5420196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:38:27.344 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f5424bb9700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5418007ae0 tx=0x7f5418004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800e040 con 0x7f5420101450 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541801db50 con 0x7f5420101450 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542019e8f0 con 0x7f5420101450 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541801bc40 con 0x7f5420101450 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.339+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542019ede0 con 0x7f5420101450 2023-08-25T04:38:27.345 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.340+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420107940 con 0x7f5420101450 2023-08-25T04:38:27.347 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.341+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) v1 ==== 134616+0+0 (secure 0 0 0) 0x7f541801dcc0 con 0x7f5420101450 2023-08-25T04:38:27.348 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.342+0000 7f541d7fa700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 0x7f54080a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:27.348 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.342+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f54180cbc50 con 0x7f5420101450 2023-08-25T04:38:27.348 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.342+0000 7f541ffff700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 0x7f54080a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:27.348 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.343+0000 7f541ffff700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 0x7f54080a2380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5420100910 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:27.348 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.345+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f54180972c0 con 0x7f5420101450 2023-08-25T04:38:27.542 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:27.540+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f542002ce70 con 0x7f540809fec0 2023-08-25T04:38:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:38:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:38:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] boot 2023-08-25T04:38:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T04:38:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:28.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:28.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:28.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] boot 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: osd.1 [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] boot 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: osdmap e13: 2 total, 2 up, 2 in 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:28.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:28.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:28 smithi183 ceph-mon[131414]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:28.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:28 smithi183 ceph-mon[131414]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:28.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:28 smithi183 ceph-mon[131414]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T04:38:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:28 smithi145 ceph-mon[97681]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:28 smithi145 ceph-mon[97681]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:28 smithi145 ceph-mon[97681]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T04:38:29.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:28 smithi162 ceph-mon[130941]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:29.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:28 smithi162 ceph-mon[130941]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-25T04:38:29.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:28 smithi162 ceph-mon[130941]: osdmap e14: 2 total, 2 up, 2 in 2023-08-25T04:38:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/1956530202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]': finished 2023-08-25T04:38:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T04:38:30.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/1956530202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]': finished 2023-08-25T04:38:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T04:38:30.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/1956530202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]: dispatch 2023-08-25T04:38:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "341bc527-6a20-413a-b07c-633058761811"}]': finished 2023-08-25T04:38:30.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: osdmap e15: 3 total, 2 up, 3 in 2023-08-25T04:38:30.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:31.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:31 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/3664762294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:31.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:31 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/3664762294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:31.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:31 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/3664762294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:32 smithi145 ceph-mon[97681]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:32.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:32 smithi162 ceph-mon[130941]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:32.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:32 smithi183 ceph-mon[131414]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:34.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:34 smithi183 ceph-mon[131414]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:35.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:34 smithi145 ceph-mon[97681]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:34 smithi162 ceph-mon[130941]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:37.027 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:36 smithi183 ceph-mon[131414]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:37.028 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T04:38:37.028 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:37.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:36 smithi145 ceph-mon[97681]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T04:38:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:37.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:36 smithi162 ceph-mon[130941]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:37.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-25T04:38:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:37 smithi183 ceph-mon[131414]: Deploying daemon osd.2 on smithi183 2023-08-25T04:38:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:37 smithi145 ceph-mon[97681]: Deploying daemon osd.2 on smithi183 2023-08-25T04:38:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:37 smithi162 ceph-mon[130941]: Deploying daemon osd.2 on smithi183 2023-08-25T04:38:38.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:38 smithi183 ceph-mon[131414]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:38 smithi145 ceph-mon[97681]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:39.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:38 smithi162 ceph-mon[130941]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:40.819 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:40 smithi183 ceph-mon[131414]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:40.819 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.819 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.819 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:40.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:40 smithi145 ceph-mon[97681]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:40.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:40 smithi162 ceph-mon[130941]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:40.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:40.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:42.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:42.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:42.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:42.834 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:42.834 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:42.834 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:42.956 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.955+0000 7f541d7fa700 1 -- 172.21.15.183:0/1546516522 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f542002ce70 con 0x7f540809fec0 2023-08-25T04:38:42.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2023-08-25T04:38:42.958 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 msgr2=0x7f54080a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:42.958 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 0x7f54080a2380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5420100910 tx=0x7f5410008040 comp rx=0 tx=0).stop 2023-08-25T04:38:42.958 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 msgr2=0x7f542019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:42.958 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5418007ae0 tx=0x7f5418004db0 comp rx=0 tx=0).stop 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 shutdown_connections 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f540809fec0 0x7f54080a2380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5420101450 0x7f542019ddd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54200ff8b0 0x7f5420196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 --2- 172.21.15.183:0/1546516522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5420100ab0 0x7f5420196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 >> 172.21.15.183:0/1546516522 conn(0x7f54200fae30 msgr2=0x7f54200fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:42.959 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 shutdown_connections 2023-08-25T04:38:42.960 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:38:42.958+0000 7f542661c700 1 -- 172.21.15.183:0/1546516522 wait complete. 2023-08-25T04:38:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:43.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:43.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:43.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:43.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:43.338 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.2.service 2023-08-25T04:38:43.342 INFO:tasks.cephadm:Deploying osd.3 on smithi145 with /dev/vg_nvme/lv_3... 2023-08-25T04:38:43.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T04:38:43.377 INFO:journalctl@ceph.osd.2.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:38:44.075 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:38:44.109 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.109 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.109 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:44.315 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.315 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.316 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:38:44.673 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T04:38:44.372+0000 7f92fa2b4640 -1 osd.2 0 log_to_monitors true 2023-08-25T04:38:45.241 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:44 smithi145 ceph-mon[97681]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:45.242 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:44 smithi145 ceph-mon[97681]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.242 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:44 smithi145 ceph-mon[97681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:44 smithi162 ceph-mon[130941]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:45.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:44 smithi162 ceph-mon[130941]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:44 smithi162 ceph-mon[130941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:44 smithi183 ceph-mon[131414]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:45.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:44 smithi183 ceph-mon[131414]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:44 smithi183 ceph-mon[131414]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-25T04:38:45.980 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T04:38:45.981 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T04:38:45.981 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2023-08-25T04:38:45.981 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2023-08-25T04:38:45.981 INFO:teuthology.orchestra.run.smithi145.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0468658 s, 224 MB/s 2023-08-25T04:38:45.981 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2023-08-25T04:38:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T04:38:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T04:38:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:46.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:45 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:46.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:46.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:46.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:45 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.422 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T04:38:45.974+0000 7f92ef92b700 -1 osd.2 0 waiting for initial osdmap 2023-08-25T04:38:46.422 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T04:38:45.978+0000 7f92e8f1b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:38:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-25T04:38:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: osdmap e16: 3 total, 2 up, 3 in 2023-08-25T04:38:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:38:46.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:45 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:46.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2023-08-25T04:38:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:38:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi183 2023-08-25T04:38:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 17168M 2023-08-25T04:38:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:38:47.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T04:38:47.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:47.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:47.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi183 2023-08-25T04:38:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 17168M 2023-08-25T04:38:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:38:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T04:38:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:47.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:47.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi183 2023-08-25T04:38:47.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 17168M 2023-08-25T04:38:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:38:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: osdmap e17: 3 total, 2 up, 3 in 2023-08-25T04:38:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.419 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:38:48.420 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:38:48.420 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.420 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] boot 2023-08-25T04:38:48.421 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T04:38:48.421 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.421 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:48 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:48.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] boot 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:48 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:48.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:38:48.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:38:48.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: osd.2 [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] boot 2023-08-25T04:38:48.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: osdmap e18: 3 total, 3 up, 3 in 2023-08-25T04:38:48.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:38:48.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:48 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:48.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 -- 172.21.15.145:0/285321615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 msgr2=0x7f3400100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 --2- 172.21.15.145:0/285321615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f3400100dd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f33f00099c0 tx=0x7f33f0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 -- 172.21.15.145:0/285321615 shutdown_connections 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 --2- 172.21.15.145:0/285321615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400102550 0x7f3400108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 --2- 172.21.15.145:0/285321615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f3400100dd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 --2- 172.21.15.145:0/285321615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400101bb0 0x7f3400102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 -- 172.21.15.145:0/285321615 >> 172.21.15.145:0/285321615 conn(0x7f34000fbf10 msgr2=0x7f34000fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:38:48.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 -- 172.21.15.145:0/285321615 shutdown_connections 2023-08-25T04:38:48.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.671+0000 7f3404b32700 1 -- 172.21.15.145:0/285321615 wait complete. 2023-08-25T04:38:48.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.672+0000 7f3404b32700 1 Processor -- start 2023-08-25T04:38:48.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 -- start start 2023-08-25T04:38:48.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:48.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 0x7f3400072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:48.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 0x7f3400073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:48.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000739d0 con 0x7f34001009b0 2023-08-25T04:38:48.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400073b40 con 0x7f3400102550 2023-08-25T04:38:48.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f3404b32700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34001a06e0 con 0x7f3400101bb0 2023-08-25T04:38:48.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f33fe59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:48.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f33fe59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:33724/0 (socket says 172.21.15.145:33724) 2023-08-25T04:38:48.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f33fe59c700 1 -- 172.21.15.145:0/2213699868 learned_addr learned my addr 172.21.15.145:0/2213699868 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:38:48.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f33fdd9b700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 0x7f3400072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:48.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.673+0000 7f33fed9d700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 0x7f3400073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:48.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 -- 172.21.15.145:0/2213699868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 msgr2=0x7f3400072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:48.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 0x7f3400072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 -- 172.21.15.145:0/2213699868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 msgr2=0x7f3400073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:38:48.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 0x7f3400073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:38:48.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 -- 172.21.15.145:0/2213699868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00095f0 con 0x7f34001009b0 2023-08-25T04:38:48.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fed9d700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 0x7f3400073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:38:48.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fdd9b700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 0x7f3400072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:38:48.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33fe59c700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009fd0 tx=0x7f33f0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:48.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f001d070 con 0x7f34001009b0 2023-08-25T04:38:48.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f0005030 con 0x7f34001009b0 2023-08-25T04:38:48.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34001a0960 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0004b30 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.674+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001a0d70 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.675+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340010c1b0 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.677+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 15) v1 ==== 134616+0+0 (secure 0 0 0) 0x7f33f00051a0 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.679+0000 7f33ef7fe700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 0x7f33e80a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.679+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f33f00caf50 con 0x7f34001009b0 2023-08-25T04:38:48.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.679+0000 7f33fdd9b700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 0x7f33e80a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:38:48.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.679+0000 7f33fdd9b700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 0x7f33e80a2380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011490 tx=0x7f33f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:38:48.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.681+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f33f0096230 con 0x7f34001009b0 2023-08-25T04:38:48.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:48.887+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3400060d60 con 0x7f33e809fec0 2023-08-25T04:38:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T04:38:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T04:38:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:49.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:49.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:49.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-25T04:38:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: osdmap e19: 3 total, 3 up, 3 in 2023-08-25T04:38:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T04:38:49.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:38:49.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:38:49.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:38:49.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:38:50.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:50.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:50 smithi145 ceph-mon[97681]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T04:38:50.524 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:50 smithi145 ceph-mon[97681]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:50.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:50.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:50 smithi162 ceph-mon[130941]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T04:38:50.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:50 smithi162 ceph-mon[130941]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:50.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-25T04:38:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:50 smithi183 ceph-mon[131414]: osdmap e20: 3 total, 3 up, 3 in 2023-08-25T04:38:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:50 smithi183 ceph-mon[131414]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:51.093 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113850]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:51.094 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113850]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:51.094 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113850]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:51.094 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113850]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:51.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:51 smithi145 ceph-mon[97681]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T04:38:51.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:51 smithi162 ceph-mon[130941]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T04:38:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:51 smithi183 ceph-mon[131414]: osdmap e21: 3 total, 3 up, 3 in 2023-08-25T04:38:52.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113888]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:52.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:52.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113888]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:52.078 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 04:38:51 smithi145 sudo[113888]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:52.147 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:51 smithi162 sudo[138304]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:52.148 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:51 smithi162 sudo[138304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:52.148 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:51 smithi162 sudo[138304]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:52.148 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:51 smithi162 sudo[138304]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:52.411 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:52 smithi162 sudo[138307]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:52.412 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:52 smithi162 sudo[138307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:52.412 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:52 smithi162 sudo[138307]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:52.412 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 04:38:52 smithi162 sudo[138307]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: osdmap e22: 3 total, 3 up, 3 in 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]: dispatch 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]': finished 2023-08-25T04:38:52.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T04:38:52.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:38:52.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: osdmap e22: 3 total, 3 up, 3 in 2023-08-25T04:38:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]: dispatch 2023-08-25T04:38:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]': finished 2023-08-25T04:38:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T04:38:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:38:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:52.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:38:52.662+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f33f0096600 con 0x7f34001009b0 2023-08-25T04:38:52.673 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139419]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:52.673 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:52.673 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139419]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:52.674 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139419]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:52.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: osdmap e22: 3 total, 3 up, 3 in 2023-08-25T04:38:52.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:52.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:52.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]: dispatch 2023-08-25T04:38:52.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3970484098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d77c0b06-c1f7-4af2-ab8b-77c462d5cd39"}]': finished 2023-08-25T04:38:52.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: osdmap e23: 4 total, 3 up, 4 in 2023-08-25T04:38:52.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:38:52.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:53.172 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139422]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:53.173 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139422]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:53.173 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139422]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:53.173 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 04:38:52 smithi183 sudo[139422]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:53.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[113977]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:53.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[113977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:53.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[113977]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:53.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[113977]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:54.008 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/849603303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:54.008 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 ceph-mon[131414]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T04:38:54.008 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:54.008 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:54.008 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 sudo[139425]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:54.009 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 sudo[139425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:54.009 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:53 smithi183 sudo[139425]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:54.009 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 sudo[139425]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[114351]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[114351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[114351]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 sudo[114351]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/849603303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 ceph-mon[97681]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T04:38:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:54.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:53 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:53 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/849603303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:38:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:53 smithi162 ceph-mon[130941]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-25T04:38:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:53 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:38:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:53 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:54.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 sudo[139428]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:54.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 sudo[139428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:54.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 sudo[139428]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:54.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 sudo[139428]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:54.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138310]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-25T04:38:54.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:54.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138310]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:54.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138310]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:55.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:55.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:54 smithi145 ceph-mon[97681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138313]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-25T04:38:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-25T04:38:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138313]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-25T04:38:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 sudo[138313]: pam_unix(sudo:session): session closed for user root 2023-08-25T04:38:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:54 smithi162 ceph-mon[130941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:38:55.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-25T04:38:55.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:38:55.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:38:55.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:54 smithi183 ceph-mon[131414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-25T04:38:57.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:56 smithi145 ceph-mon[97681]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:57.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:56 smithi162 ceph-mon[130941]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:57.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:56 smithi183 ceph-mon[131414]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:59.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:58 smithi145 ceph-mon[97681]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:59.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:58 smithi162 ceph-mon[130941]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:38:59.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:58 smithi183 ceph-mon[131414]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:39:00.070 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:59 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T04:39:00.071 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:38:59 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:59 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T04:39:00.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:38:59 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:59 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-25T04:39:00.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:38:59 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:01.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:00 smithi145 ceph-mon[97681]: Deploying daemon osd.3 on smithi145 2023-08-25T04:39:01.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:00 smithi145 ceph-mon[97681]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-08-25T04:39:01.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:00 smithi162 ceph-mon[130941]: Deploying daemon osd.3 on smithi145 2023-08-25T04:39:01.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:00 smithi162 ceph-mon[130941]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-08-25T04:39:01.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:00 smithi183 ceph-mon[131414]: Deploying daemon osd.3 on smithi145 2023-08-25T04:39:01.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:00 smithi183 ceph-mon[131414]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-08-25T04:39:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:02 smithi145 ceph-mon[97681]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-08-25T04:39:03.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:02 smithi162 ceph-mon[130941]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-08-25T04:39:03.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:02 smithi183 ceph-mon[131414]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-08-25T04:39:05.046 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:04 smithi145 ceph-mon[97681]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T04:39:05.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:04 smithi162 ceph-mon[130941]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T04:39:05.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:04 smithi183 ceph-mon[131414]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-08-25T04:39:06.468 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.468 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.468 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:06 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:06.469 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:06 smithi145 ceph-mon[97681]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:06.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:06 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:06.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:06 smithi162 ceph-mon[130941]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:06.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:06.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:06 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:06.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:06 smithi183 ceph-mon[131414]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:08.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.496+0000 7f33ef7fe700 1 -- 172.21.15.145:0/2213699868 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3400060d60 con 0x7f33e809fec0 2023-08-25T04:39:08.500 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 3 on host 'smithi145' 2023-08-25T04:39:08.502 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 msgr2=0x7f33e80a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:08.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 0x7f33e80a2380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011490 tx=0x7f33f4009260 comp rx=0 tx=0).stop 2023-08-25T04:39:08.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 msgr2=0x7f34000746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:08.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009fd0 tx=0x7f33f0009580 comp rx=0 tx=0).stop 2023-08-25T04:39:08.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 shutdown_connections 2023-08-25T04:39:08.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f33e809fec0 0x7f33e80a2380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:08.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3400102550 0x7f3400073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:08.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34001009b0 0x7f34000746c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:08.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 --2- 172.21.15.145:0/2213699868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3400101bb0 0x7f3400072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:08.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 >> 172.21.15.145:0/2213699868 conn(0x7f34000fbf10 msgr2=0x7f34000fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:08.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 shutdown_connections 2023-08-25T04:39:08.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:08.500+0000 7f3404b32700 1 -- 172.21.15.145:0/2213699868 wait complete. 2023-08-25T04:39:08.904 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:08.905 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:08.905 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:08.905 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:08.905 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:08.906 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:08.906 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:08.906 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:08.906 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:08 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:08.907 DEBUG:teuthology.orchestra.run.smithi145:osd.3> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.3.service 2023-08-25T04:39:08.912 INFO:tasks.cephadm:Deploying osd.4 on smithi162 with /dev/vg_nvme/lv_3... 2023-08-25T04:39:08.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T04:39:08.982 INFO:journalctl@ceph.osd.3.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:39:09.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:08 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:09.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:09.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:08 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:39:09.527 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 04:39:09 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T04:39:09.168+0000 7efde49d8640 -1 osd.3 0 log_to_monitors true 2023-08-25T04:39:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:09 smithi162 ceph-mon[130941]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:09 smithi162 ceph-mon[130941]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:09 smithi145 ceph-mon[97681]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:09 smithi145 ceph-mon[97681]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:09 smithi183 ceph-mon[131414]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:09 smithi183 ceph-mon[131414]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-25T04:39:10.849 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T04:39:10.850 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T04:39:10.850 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2023-08-25T04:39:10.850 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2023-08-25T04:39:10.850 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110892 s, 946 MB/s 2023-08-25T04:39:10.851 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2023-08-25T04:39:11.078 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T04:39:10.821+0000 7efdda04f700 -1 osd.3 0 waiting for initial osdmap 2023-08-25T04:39:11.079 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T04:39:10.830+0000 7efdd263d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:39:11.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:11.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T04:39:11.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T04:39:11.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:11.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:10 smithi145 ceph-mon[97681]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T04:39:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T04:39:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:10 smithi162 ceph-mon[130941]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-08-25T04:39:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-25T04:39:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: osdmap e24: 4 total, 3 up, 4 in 2023-08-25T04:39:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: from='osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:10 smithi183 ceph-mon[131414]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:39:11.216 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2023-08-25T04:39:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:39:12.157 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:39:12.157 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: osdmap e25: 4 total, 3 up, 4 in 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.158 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.159 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:12.159 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:12.159 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: osdmap e25: 4 total, 3 up, 4 in 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:12.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:39:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: osdmap e25: 4 total, 3 up, 4 in 2023-08-25T04:39:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:12.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:12.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:12.961 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 -- 172.21.15.162:0/1683432512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59240ff230 msgr2=0x7f59240ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:12.961 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 --2- 172.21.15.162:0/1683432512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59240ff230 0x7f59240ff650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5914009990 tx=0x7f5914009ca0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 -- 172.21.15.162:0/1683432512 shutdown_connections 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 --2- 172.21.15.162:0/1683432512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5924100550 0x7f59241009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 --2- 172.21.15.162:0/1683432512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59240ff230 0x7f59240ff650 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 --2- 172.21.15.162:0/1683432512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ffb90 0x7f5924100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 -- 172.21.15.162:0/1683432512 >> 172.21.15.162:0/1683432512 conn(0x7f59240fae30 msgr2=0x7f59240fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:12.962 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 -- 172.21.15.162:0/1683432512 shutdown_connections 2023-08-25T04:39:12.963 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.960+0000 7f592b7e0700 1 -- 172.21.15.162:0/1683432512 wait complete. 2023-08-25T04:39:12.965 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.961+0000 7f592b7e0700 1 Processor -- start 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.961+0000 7f592b7e0700 1 -- start start 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.961+0000 7f592b7e0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 0x7f5924198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.961+0000 7f592b7e0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59240ffb90 0x7f5924198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.961+0000 7f592b7e0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592b7e0700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59241a04a0 con 0x7f59240ffb90 2023-08-25T04:39:12.966 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592b7e0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59241a05e0 con 0x7f5924100550 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592b7e0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59241a0770 con 0x7f59240ff230 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33030/0 (socket says 172.21.15.162:33030) 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 -- 172.21.15.162:0/3699113465 learned_addr learned my addr 172.21.15.162:0/3699113465 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592957c700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 0x7f5924198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:12.967 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 -- 172.21.15.162:0/3699113465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 msgr2=0x7f5924198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 0x7f5924198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 -- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59240ffb90 msgr2=0x7f5924198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59240ffb90 0x7f5924198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 -- 172.21.15.162:0/3699113465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140095f0 con 0x7f5924100550 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592957c700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 0x7f5924198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f5929d7d700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b970 tx=0x7f591c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:12.968 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00c9b0 con 0x7f5924100550 2023-08-25T04:39:12.969 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59241a0a50 con 0x7f5924100550 2023-08-25T04:39:12.969 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c014440 con 0x7f5924100550 2023-08-25T04:39:12.969 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0075d0 con 0x7f5924100550 2023-08-25T04:39:12.969 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.962+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59241a0f80 con 0x7f5924100550 2023-08-25T04:39:12.970 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.963+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592410a010 con 0x7f5924100550 2023-08-25T04:39:12.970 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.964+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f591c0148d0 con 0x7f5924100550 2023-08-25T04:39:12.971 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.965+0000 7f591a7fc700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 0x7f59100a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:12.971 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.965+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f591c006e40 con 0x7f5924100550 2023-08-25T04:39:12.971 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.966+0000 7f592957c700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 0x7f59100a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:12.972 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.966+0000 7f592957c700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 0x7f59100a2410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f591400b5c0 tx=0x7f5914005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:12.972 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:12.969+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f591c094b30 con 0x7f5924100550 2023-08-25T04:39:13.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:39:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:39:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi145 2023-08-25T04:39:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 8633M 2023-08-25T04:39:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:39:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] boot 2023-08-25T04:39:13.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T04:39:13.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:13.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:12 smithi162 ceph-mon[130941]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T04:39:13.166 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:13.163+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f59240008d0 con 0x7f591009ff50 2023-08-25T04:39:13.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:39:13.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi145 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 8633M 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] boot 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:12 smithi183 ceph-mon[131414]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T04:39:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:39:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:39:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi145 2023-08-25T04:39:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 8633M 2023-08-25T04:39:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-25T04:39:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: osd.3 [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] boot 2023-08-25T04:39:13.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: osdmap e26: 4 total, 4 up, 4 in 2023-08-25T04:39:13.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:39:13.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:12 smithi145 ceph-mon[97681]: osdmap e27: 4 total, 4 up, 4 in 2023-08-25T04:39:14.143 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:13 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:14.144 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:13 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:14.144 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:13 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:14.144 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:13 smithi162 ceph-mon[130941]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T04:39:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:13 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:14.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:13 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:14.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:13 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:14.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:13 smithi183 ceph-mon[131414]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T04:39:14.341 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:13 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:14.341 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:13 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:14.341 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:13 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:14.341 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:13 smithi145 ceph-mon[97681]: osdmap e28: 4 total, 4 up, 4 in 2023-08-25T04:39:15.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:14 smithi162 ceph-mon[130941]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:14 smithi162 ceph-mon[130941]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:14 smithi162 ceph-mon[130941]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T04:39:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:14 smithi183 ceph-mon[131414]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:14 smithi183 ceph-mon[131414]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:14 smithi183 ceph-mon[131414]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T04:39:15.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:14 smithi145 ceph-mon[97681]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:15.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:14 smithi145 ceph-mon[97681]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:15.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:14 smithi145 ceph-mon[97681]: osdmap e29: 4 total, 4 up, 4 in 2023-08-25T04:39:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:15 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/1124072898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:15 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:15 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]': finished 2023-08-25T04:39:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:15 smithi162 ceph-mon[130941]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T04:39:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:15 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:15 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/1124072898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:15 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:15 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]': finished 2023-08-25T04:39:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:15 smithi183 ceph-mon[131414]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T04:39:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:15 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:16.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:15 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/1124072898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:15 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]: dispatch 2023-08-25T04:39:16.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:15 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f547bc71-6bd5-4882-baa1-d043181e9ff8"}]': finished 2023-08-25T04:39:16.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:15 smithi145 ceph-mon[97681]: osdmap e30: 5 total, 4 up, 5 in 2023-08-25T04:39:16.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:15 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:16 smithi162 ceph-mon[130941]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:16 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/854334965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:16 smithi183 ceph-mon[131414]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:16 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/854334965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:17.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:16 smithi145 ceph-mon[97681]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:17.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:16 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/854334965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:18 smithi162 ceph-mon[130941]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:18 smithi183 ceph-mon[131414]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:19.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:18 smithi145 ceph-mon[97681]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:20.585 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:20 smithi162 ceph-mon[130941]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T04:39:20.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:20 smithi145 ceph-mon[97681]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T04:39:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:20 smithi183 ceph-mon[131414]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-25T04:39:22.794 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:22 smithi162 ceph-mon[130941]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-08-25T04:39:22.794 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T04:39:22.794 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:22.794 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:22 smithi162 ceph-mon[130941]: Deploying daemon osd.4 on smithi162 2023-08-25T04:39:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:22 smithi145 ceph-mon[97681]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-08-25T04:39:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T04:39:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:22 smithi145 ceph-mon[97681]: Deploying daemon osd.4 on smithi162 2023-08-25T04:39:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:22 smithi183 ceph-mon[131414]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-08-25T04:39:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-25T04:39:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:22 smithi183 ceph-mon[131414]: Deploying daemon osd.4 on smithi162 2023-08-25T04:39:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:24 smithi145 ceph-mon[97681]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:39:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:24 smithi162 ceph-mon[130941]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:39:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:24 smithi183 ceph-mon[131414]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:39:26.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:26 smithi162 ceph-mon[130941]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T04:39:26.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:26.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:26.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:27.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:26 smithi145 ceph-mon[97681]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T04:39:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:27.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:26 smithi183 ceph-mon[131414]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-08-25T04:39:27.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:27.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:27.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.676 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.674+0000 7f591a7fc700 1 -- 172.21.15.162:0/3699113465 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59240008d0 con 0x7f591009ff50 2023-08-25T04:39:28.676 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 4 on host 'smithi162' 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 msgr2=0x7f59100a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 0x7f59100a2410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f591400b5c0 tx=0x7f5914005fb0 comp rx=0 tx=0).stop 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 msgr2=0x7f592419fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b970 tx=0x7f591c00bd30 comp rx=0 tx=0).stop 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 shutdown_connections 2023-08-25T04:39:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f591009ff50 0x7f59100a2410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5924100550 0x7f592419fed0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59240ff230 0x7f5924198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 --2- 172.21.15.162:0/3699113465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59240ffb90 0x7f5924198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 >> 172.21.15.162:0/3699113465 conn(0x7f59240fae30 msgr2=0x7f59240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 shutdown_connections 2023-08-25T04:39:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:39:28.677+0000 7f592b7e0700 1 -- 172.21.15.162:0/3699113465 wait complete. 2023-08-25T04:39:28.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:28.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:28.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:28.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:28.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:28 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:29.049 DEBUG:teuthology.orchestra.run.smithi162:osd.4> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.4.service 2023-08-25T04:39:29.055 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2023-08-25T04:39:29.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-25T04:39:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:28 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:29.086 INFO:journalctl@ceph.osd.4.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:39:29.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:29.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.088 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:29.089 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:29.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.090 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.091 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:29.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:28 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:29.619 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:39:30.412 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 04:39:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T04:39:30.150+0000 7fe874978640 -1 osd.4 0 log_to_monitors true 2023-08-25T04:39:30.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:30 smithi183 ceph-mon[131414]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:30.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:30 smithi183 ceph-mon[131414]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:30 smithi183 ceph-mon[131414]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.807 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:30 smithi162 ceph-mon[130941]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:30.808 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:30 smithi162 ceph-mon[130941]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.808 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:30 smithi162 ceph-mon[130941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:30 smithi145 ceph-mon[97681]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:39:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:30 smithi145 ceph-mon[97681]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:30 smithi145 ceph-mon[97681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-25T04:39:30.990 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-25T04:39:30.990 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-25T04:39:30.991 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2023-08-25T04:39:30.991 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2023-08-25T04:39:30.991 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111289 s, 942 MB/s 2023-08-25T04:39:30.991 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2023-08-25T04:39:31.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2023-08-25T04:39:31.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T04:39:31.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T04:39:31.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:31.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:31.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:31.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T04:39:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T04:39:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:31.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:31.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.911 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T04:39:31.488+0000 7fe869fef700 -1 osd.4 0 waiting for initial osdmap 2023-08-25T04:39:31.912 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T04:39:31.493+0000 7fe861ddc700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:39:31.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-25T04:39:31.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: osdmap e31: 5 total, 4 up, 5 in 2023-08-25T04:39:31.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:31.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:39:31.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:31.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:31.925 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:39:32.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi162 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 8584M 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:32 smithi183 ceph-mon[131414]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:32.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi162 2023-08-25T04:39:32.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 8584M 2023-08-25T04:39:32.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:39:32.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T04:39:32.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:32 smithi145 ceph-mon[97681]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:32.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi162 2023-08-25T04:39:32.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 8584M 2023-08-25T04:39:32.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:39:32.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: osdmap e32: 5 total, 4 up, 5 in 2023-08-25T04:39:32.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:32.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:32 smithi162 ceph-mon[130941]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-08-25T04:39:33.006 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- 172.21.15.183:0/4064792924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 msgr2=0x7f00a8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:33.006 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 --2- 172.21.15.183:0/4064792924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a8108e00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f00a000b0a0 tx=0x7f00a000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- 172.21.15.183:0/4064792924 shutdown_connections 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 --2- 172.21.15.183:0/4064792924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a8108e00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 --2- 172.21.15.183:0/4064792924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a81009b0 0x7f00a8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 --2- 172.21.15.183:0/4064792924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a8101bb0 0x7f00a8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- 172.21.15.183:0/4064792924 >> 172.21.15.183:0/4064792924 conn(0x7f00a80fbf10 msgr2=0x7f00a80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:33.007 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- 172.21.15.183:0/4064792924 shutdown_connections 2023-08-25T04:39:33.008 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- 172.21.15.183:0/4064792924 wait complete. 2023-08-25T04:39:33.008 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 Processor -- start 2023-08-25T04:39:33.009 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.002+0000 7f00ae45f700 1 -- start start 2023-08-25T04:39:33.009 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 0x7f00a8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a8101bb0 0x7f00a8196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a819e2d0 con 0x7f00a81009b0 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a819e410 con 0x7f00a8101bb0 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ae45f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a819e5a0 con 0x7f00a8102550 2023-08-25T04:39:33.010 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52224/0 (socket says 172.21.15.183:52224) 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 -- 172.21.15.183:0/1515852785 learned_addr learned my addr 172.21.15.183:0/1515852785 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00a77fe700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a8101bb0 0x7f00a8196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 -- 172.21.15.183:0/1515852785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a8101bb0 msgr2=0x7f00a8196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00a7fff700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 0x7f00a8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a8101bb0 0x7f00a8196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.011 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 -- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 msgr2=0x7f00a8196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:33.012 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 0x7f00a8196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:33.012 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 -- 172.21.15.183:0/1515852785 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0009d00 con 0x7f00a8102550 2023-08-25T04:39:33.012 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00a7fff700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 0x7f00a8196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:39:33.012 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.003+0000 7f00ac9fc700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004d00 tx=0x7f00a0004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:33.012 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.004+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000e040 con 0x7f00a8102550 2023-08-25T04:39:33.013 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.004+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00a001bd20 con 0x7f00a8102550 2023-08-25T04:39:33.013 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.004+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a819e820 con 0x7f00a8102550 2023-08-25T04:39:33.013 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.004+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0005920 con 0x7f00a8102550 2023-08-25T04:39:33.013 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.004+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a819ed60 con 0x7f00a8102550 2023-08-25T04:39:33.013 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.005+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a8107e50 con 0x7f00a8102550 2023-08-25T04:39:33.015 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.006+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f00a0005040 con 0x7f00a8102550 2023-08-25T04:39:33.015 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.006+0000 7f00a57fa700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 0x7f00900a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:33.015 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.006+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f00a00cbec0 con 0x7f00a8102550 2023-08-25T04:39:33.015 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.007+0000 7f00a7fff700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 0x7f00900a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:33.015 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.007+0000 7f00a7fff700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 0x7f00900a2490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0098005fd0 tx=0x7f0098005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:33.016 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.011+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f00a0096d50 con 0x7f00a8102550 2023-08-25T04:39:33.207 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:33.206+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f00a802cda0 con 0x7f009009ffd0 2023-08-25T04:39:33.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:39:33.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] boot 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:33.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:39:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] boot 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:33.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:33.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:39:33.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:39:33.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: osd.4 [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] boot 2023-08-25T04:39:33.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: osdmap e33: 5 total, 5 up, 5 in 2023-08-25T04:39:33.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:39:33.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:33.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:33.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:34.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:34 smithi183 ceph-mon[131414]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:34.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:34 smithi183 ceph-mon[131414]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T04:39:34.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:34 smithi183 ceph-mon[131414]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:34.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:34 smithi145 ceph-mon[97681]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:34.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:34 smithi145 ceph-mon[97681]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T04:39:34.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:34 smithi145 ceph-mon[97681]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:34.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:34 smithi162 ceph-mon[130941]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:34.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:34 smithi162 ceph-mon[130941]: osdmap e34: 5 total, 5 up, 5 in 2023-08-25T04:39:34.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:34 smithi162 ceph-mon[130941]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:35.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:35 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/284787242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:35 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:35 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]': finished 2023-08-25T04:39:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:35 smithi162 ceph-mon[130941]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T04:39:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:35 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/284787242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:35 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:35 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]': finished 2023-08-25T04:39:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:35 smithi183 ceph-mon[131414]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T04:39:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:35 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/284787242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:35 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]: dispatch 2023-08-25T04:39:36.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:35 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bffd6e2e-e1b7-4dee-8a67-98c02dc1162d"}]': finished 2023-08-25T04:39:36.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:35 smithi145 ceph-mon[97681]: osdmap e35: 6 total, 5 up, 6 in 2023-08-25T04:39:36.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:36.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:36 smithi162 ceph-mon[130941]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:36.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:36 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/1316749084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:36.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:36 smithi183 ceph-mon[131414]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:36.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:36 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/1316749084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:36 smithi145 ceph-mon[97681]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:36 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/1316749084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:38 smithi145 ceph-mon[97681]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:39.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:38 smithi162 ceph-mon[130941]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:38 smithi183 ceph-mon[131414]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:40.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:40 smithi183 ceph-mon[131414]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:40.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:40 smithi145 ceph-mon[97681]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:40.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:40 smithi162 ceph-mon[130941]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:42 smithi183 ceph-mon[131414]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T04:39:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:42 smithi183 ceph-mon[131414]: Deploying daemon osd.5 on smithi183 2023-08-25T04:39:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:42 smithi145 ceph-mon[97681]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T04:39:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:43.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:42 smithi145 ceph-mon[97681]: Deploying daemon osd.5 on smithi183 2023-08-25T04:39:43.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:42 smithi162 ceph-mon[130941]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-25T04:39:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:42 smithi162 ceph-mon[130941]: Deploying daemon osd.5 on smithi183 2023-08-25T04:39:44.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:44 smithi183 ceph-mon[131414]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:45.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:44 smithi145 ceph-mon[97681]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:45.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:44 smithi162 ceph-mon[130941]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:46 smithi183 ceph-mon[131414]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:46 smithi145 ceph-mon[97681]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:47.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:46 smithi162 ceph-mon[130941]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:48.768 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2023-08-25T04:39:48.769 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.766+0000 7f00a57fa700 1 -- 172.21.15.183:0/1515852785 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f00a802cda0 con 0x7f009009ffd0 2023-08-25T04:39:48.769 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 msgr2=0x7f00900a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:48.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 0x7f00900a2490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0098005fd0 tx=0x7f0098005e40 comp rx=0 tx=0).stop 2023-08-25T04:39:48.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 msgr2=0x7f00a819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:48.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004d00 tx=0x7f00a0004de0 comp rx=0 tx=0).stop 2023-08-25T04:39:48.770 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 shutdown_connections 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f009009ffd0 0x7f00900a2490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00a8102550 0x7f00a819dd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00a81009b0 0x7f00a8196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 --2- 172.21.15.183:0/1515852785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00a8101bb0 0x7f00a8196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 >> 172.21.15.183:0/1515852785 conn(0x7f00a80fbf10 msgr2=0x7f00a8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 shutdown_connections 2023-08-25T04:39:48.771 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:39:48.768+0000 7f00ae45f700 1 -- 172.21.15.183:0/1515852785 wait complete. 2023-08-25T04:39:49.113 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.5.service 2023-08-25T04:39:49.115 INFO:tasks.cephadm:Deploying osd.6 on smithi145 with /dev/vg_nvme/lv_2... 2023-08-25T04:39:49.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T04:39:49.157 INFO:journalctl@ceph.osd.5.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:39:49.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:49 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:49.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:49.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:49.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.581 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.581 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:49.581 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:49 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:39:49.863 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:39:50.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 04:39:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T04:39:49.990+0000 7f9c88e7e640 -1 osd.5 0 log_to_monitors true 2023-08-25T04:39:50.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:50 smithi145 ceph-mon[97681]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:50.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:50 smithi145 ceph-mon[97681]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:50.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:50 smithi145 ceph-mon[97681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:50.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:50 smithi162 ceph-mon[130941]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:50.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:50 smithi162 ceph-mon[130941]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:50.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:50 smithi162 ceph-mon[130941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:50.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:50 smithi183 ceph-mon[131414]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:50.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:50 smithi183 ceph-mon[131414]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:50.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:50 smithi183 ceph-mon[131414]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-25T04:39:51.672 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T04:39:51.498+0000 7f9c7e4f5700 -1 osd.5 0 waiting for initial osdmap 2023-08-25T04:39:51.673 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T04:39:51.504+0000 7f9c78ae7700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:39:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T04:39:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T04:39:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:51.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:51.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.741 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T04:39:51.742 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T04:39:51.742 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2023-08-25T04:39:51.742 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2023-08-25T04:39:51.744 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0301378 s, 348 MB/s 2023-08-25T04:39:51.744 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2023-08-25T04:39:51.779 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T04:39:51.779 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T04:39:51.779 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:51.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:51.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:51.782 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-25T04:39:51.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: osdmap e36: 6 total, 5 up, 6 in 2023-08-25T04:39:51.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:51.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:39:51.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:39:51.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:51.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:51.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:39:51.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:39:52.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2023-08-25T04:39:52.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi183 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 8584M 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:52.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi183 2023-08-25T04:39:52.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 8584M 2023-08-25T04:39:52.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:39:52.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T04:39:52.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:52.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi183 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 8584M 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: osdmap e37: 6 total, 5 up, 6 in 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-08-25T04:39:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:52.966 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:39:53.777 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:53 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:39:53.777 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:53 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:39:53.777 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:53 smithi145 ceph-mon[97681]: osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] boot 2023-08-25T04:39:53.778 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:53 smithi145 ceph-mon[97681]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T04:39:53.778 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:53 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:53.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:53 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:39:53.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:53 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:39:53.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:53 smithi162 ceph-mon[130941]: osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] boot 2023-08-25T04:39:53.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:53 smithi162 ceph-mon[130941]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T04:39:53.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:53 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:53.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:53 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:39:53.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:53 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:39:53.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:53 smithi183 ceph-mon[131414]: osd.5 [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] boot 2023-08-25T04:39:53.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:53 smithi183 ceph-mon[131414]: osdmap e38: 6 total, 6 up, 6 in 2023-08-25T04:39:53.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:53 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 -- 172.21.15.145:0/2476862553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c102550 msgr2=0x7fd23c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 --2- 172.21.15.145:0/2476862553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c102550 0x7fd23c108e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd23400b0d0 tx=0x7fd23400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 -- 172.21.15.145:0/2476862553 shutdown_connections 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 --2- 172.21.15.145:0/2476862553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c102550 0x7fd23c108e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 --2- 172.21.15.145:0/2476862553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c1009b0 0x7fd23c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.348+0000 7fd24389f700 1 --2- 172.21.15.145:0/2476862553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 0x7fd23c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.349+0000 7fd24389f700 1 -- 172.21.15.145:0/2476862553 >> 172.21.15.145:0/2476862553 conn(0x7fd23c0fbf10 msgr2=0x7fd23c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:39:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.349+0000 7fd24389f700 1 -- 172.21.15.145:0/2476862553 shutdown_connections 2023-08-25T04:39:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.349+0000 7fd24389f700 1 -- 172.21.15.145:0/2476862553 wait complete. 2023-08-25T04:39:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 Processor -- start 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 -- start start 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c1009b0 0x7fd23c196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 0x7fd23c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c19e2e0 con 0x7fd23c1009b0 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c19e420 con 0x7fd23c101bb0 2023-08-25T04:39:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24389f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c19e5b0 con 0x7fd23c102550 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd24163b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c1009b0 0x7fd23c196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd241e3c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd241e3c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42740/0 (socket says 172.21.15.145:42740) 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.350+0000 7fd241e3c700 1 -- 172.21.15.145:0/1747843509 learned_addr learned my addr 172.21.15.145:0/1747843509 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd240e3a700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 0x7fd23c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 -- 172.21.15.145:0/1747843509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 msgr2=0x7fd23c196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 0x7fd23c196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 -- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c1009b0 msgr2=0x7fd23c196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c1009b0 0x7fd23c196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 -- 172.21.15.145:0/1747843509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd234009d00 con 0x7fd23c102550 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd241e3c700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd234008f40 tx=0x7fd234008f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400e040 con 0x7fd23c102550 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.351+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c072c10 con 0x7fd23c102550 2023-08-25T04:39:54.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.352+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c073100 con 0x7fd23c102550 2023-08-25T04:39:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.352+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23400be30 con 0x7fd23c102550 2023-08-25T04:39:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.352+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23401b6c0 con 0x7fd23c102550 2023-08-25T04:39:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.353+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd220005320 con 0x7fd23c102550 2023-08-25T04:39:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.355+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd234003c00 con 0x7fd23c102550 2023-08-25T04:39:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.356+0000 7fd2327fc700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 0x7fd2280a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:39:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.356+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fd2340cb590 con 0x7fd23c102550 2023-08-25T04:39:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.356+0000 7fd24163b700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 0x7fd2280a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:39:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.357+0000 7fd24163b700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 0x7fd2280a2490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd23c101a10 tx=0x7fd238009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:39:54.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.358+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd2340961b0 con 0x7fd23c102550 2023-08-25T04:39:54.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:39:54.572+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd220000bf0 con 0x7fd22809ffd0 2023-08-25T04:39:54.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:54 smithi145 ceph-mon[97681]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T04:39:54.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:54 smithi145 ceph-mon[97681]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:54.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:54 smithi162 ceph-mon[130941]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T04:39:54.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:54 smithi162 ceph-mon[130941]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:54.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:54 smithi183 ceph-mon[131414]: osdmap e39: 6 total, 6 up, 6 in 2023-08-25T04:39:54.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:54 smithi183 ceph-mon[131414]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:55.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:55 smithi145 ceph-mon[97681]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T04:39:55.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:55 smithi145 ceph-mon[97681]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:55.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:55 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:55.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:55 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:55.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:55 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:55.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:55 smithi162 ceph-mon[130941]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T04:39:55.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:55 smithi162 ceph-mon[130941]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:55.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:55 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:55.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:55 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:55.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:55 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:55.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:55 smithi183 ceph-mon[131414]: osdmap e40: 6 total, 6 up, 6 in 2023-08-25T04:39:55.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:55 smithi183 ceph-mon[131414]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:39:55.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:55 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:39:55.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:55 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:39:55.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:55 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:39:56.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:56 smithi145 ceph-mon[97681]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T04:39:56.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:56 smithi145 ceph-mon[97681]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:56.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:56 smithi162 ceph-mon[130941]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T04:39:56.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:56 smithi162 ceph-mon[130941]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:56.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:56 smithi183 ceph-mon[131414]: osdmap e41: 6 total, 6 up, 6 in 2023-08-25T04:39:56.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:56 smithi183 ceph-mon[131414]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T04:39:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/578947542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]': finished 2023-08-25T04:39:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T04:39:57.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:39:57.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T04:39:57.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/578947542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]': finished 2023-08-25T04:39:57.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T04:39:57.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:39:57.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: osdmap e42: 6 total, 6 up, 6 in 2023-08-25T04:39:57.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/578947542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]: dispatch 2023-08-25T04:39:57.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46c86644-5bd6-4012-8847-8e5c35e97f6c"}]': finished 2023-08-25T04:39:57.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: osdmap e43: 7 total, 6 up, 7 in 2023-08-25T04:39:57.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:39:58.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:58 smithi145 ceph-mon[97681]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:58.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:39:58 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2758818810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:58.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:58 smithi162 ceph-mon[130941]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:58.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:39:58 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2758818810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:39:58.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:58 smithi183 ceph-mon[131414]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:39:58.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:39:58 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2758818810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:00.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:00 smithi145 ceph-mon[97681]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:40:00.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:00 smithi145 ceph-mon[97681]: overall HEALTH_OK 2023-08-25T04:40:00.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:00 smithi162 ceph-mon[130941]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:40:00.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:00 smithi162 ceph-mon[130941]: overall HEALTH_OK 2023-08-25T04:40:00.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:00 smithi183 ceph-mon[131414]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:40:00.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:00 smithi183 ceph-mon[131414]: overall HEALTH_OK 2023-08-25T04:40:03.014 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:02 smithi145 ceph-mon[97681]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:40:03.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:02 smithi162 ceph-mon[130941]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:40:03.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:02 smithi183 ceph-mon[131414]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:40:05.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:04 smithi145 ceph-mon[97681]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:40:05.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:04 smithi162 ceph-mon[130941]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:40:05.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:04 smithi183 ceph-mon[131414]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:40:05.969 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:05 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T04:40:05.969 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:05 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:05.969 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:05 smithi145 ceph-mon[97681]: Deploying daemon osd.6 on smithi145 2023-08-25T04:40:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:05 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T04:40:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:05 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:05 smithi162 ceph-mon[130941]: Deploying daemon osd.6 on smithi145 2023-08-25T04:40:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:05 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-25T04:40:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:05 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:05 smithi183 ceph-mon[131414]: Deploying daemon osd.6 on smithi145 2023-08-25T04:40:07.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:06 smithi145 ceph-mon[97681]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:40:07.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:06 smithi162 ceph-mon[130941]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:40:07.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:06 smithi183 ceph-mon[131414]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-08-25T04:40:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:08 smithi145 ceph-mon[97681]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-25T04:40:09.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:08 smithi162 ceph-mon[130941]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-25T04:40:09.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:08 smithi183 ceph-mon[131414]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-08-25T04:40:10.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:10 smithi145 ceph-mon[97681]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:40:10.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:10 smithi162 ceph-mon[130941]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:40:10.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:10 smithi183 ceph-mon[131414]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:40:12.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:11 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:11 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:11 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:13.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:12 smithi145 ceph-mon[97681]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:13.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:12 smithi162 ceph-mon[130941]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:13.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:12 smithi183 ceph-mon[131414]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:13.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.890+0000 7fd2327fc700 1 -- 172.21.15.145:0/1747843509 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd220000bf0 con 0x7fd22809ffd0 2023-08-25T04:40:13.892 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 6 on host 'smithi145' 2023-08-25T04:40:13.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 msgr2=0x7fd2280a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:13.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 0x7fd2280a2490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd23c101a10 tx=0x7fd238009260 comp rx=0 tx=0).stop 2023-08-25T04:40:13.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 msgr2=0x7fd23c19dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:13.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd234008f40 tx=0x7fd234008f70 comp rx=0 tx=0).stop 2023-08-25T04:40:13.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 shutdown_connections 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd22809ffd0 0x7fd2280a2490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd23c102550 0x7fd23c19dd10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd23c1009b0 0x7fd23c196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 --2- 172.21.15.145:0/1747843509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd23c101bb0 0x7fd23c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 >> 172.21.15.145:0/1747843509 conn(0x7fd23c0fbf10 msgr2=0x7fd23c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 shutdown_connections 2023-08-25T04:40:13.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:40:13.893+0000 7fd24389f700 1 -- 172.21.15.145:0/1747843509 wait complete. 2023-08-25T04:40:14.296 DEBUG:teuthology.orchestra.run.smithi145:osd.6> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.6.service 2023-08-25T04:40:14.300 INFO:tasks.cephadm:Deploying osd.7 on smithi162 with /dev/vg_nvme/lv_2... 2023-08-25T04:40:14.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T04:40:14.370 INFO:journalctl@ceph.osd.6.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:40:14.879 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:40:14.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:15.003 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T04:40:14.800+0000 7fce245fd640 -1 osd.6 0 log_to_monitors true 2023-08-25T04:40:15.003 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:15.003 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:15.004 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:15.872 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:15 smithi162 ceph-mon[130941]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T04:40:15.915 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:15 smithi145 ceph-mon[97681]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T04:40:15.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:15 smithi183 ceph-mon[131414]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-25T04:40:16.365 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T04:40:16.366 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T04:40:16.366 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2023-08-25T04:40:16.366 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2023-08-25T04:40:16.366 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132491 s, 791 MB/s 2023-08-25T04:40:16.366 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2023-08-25T04:40:16.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2023-08-25T04:40:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-mon[97681]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-mon[97681]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T04:40:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-mon[97681]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T04:40:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-mon[97681]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:40:17.080 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T04:40:16.673+0000 7fce19c74700 -1 osd.6 0 waiting for initial osdmap 2023-08-25T04:40:17.080 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 04:40:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T04:40:16.685+0000 7fce15a69700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:40:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:16 smithi162 ceph-mon[130941]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:16 smithi162 ceph-mon[130941]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T04:40:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:16 smithi162 ceph-mon[130941]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T04:40:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:16 smithi162 ceph-mon[130941]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:40:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:16 smithi183 ceph-mon[131414]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:16 smithi183 ceph-mon[131414]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-25T04:40:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:16 smithi183 ceph-mon[131414]: osdmap e44: 7 total, 6 up, 7 in 2023-08-25T04:40:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:16 smithi183 ceph-mon[131414]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:40:17.285 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:40:17.780 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:17.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:17.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:17 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:40:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T04:40:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:18.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:18.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:17 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: osdmap e45: 7 total, 6 up, 7 in 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:18.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:18.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:18.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:17 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:18.459 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 -- 172.21.15.162:0/1963108935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0100ab0 msgr2=0x7fb9f0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/1963108935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0100f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00099c0 tx=0x7fb9e0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 -- 172.21.15.162:0/1963108935 shutdown_connections 2023-08-25T04:40:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/1963108935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0101450 0x7fb9f0107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/1963108935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 0x7fb9f00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/1963108935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0100f10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 -- 172.21.15.162:0/1963108935 >> 172.21.15.162:0/1963108935 conn(0x7fb9f00fae30 msgr2=0x7fb9f00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:40:18.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 -- 172.21.15.162:0/1963108935 shutdown_connections 2023-08-25T04:40:18.461 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.458+0000 7fb9f48c7700 1 -- 172.21.15.162:0/1963108935 wait complete. 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 Processor -- start 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 -- start start 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 0x7fb9f01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019d180 con 0x7fb9f0100ab0 2023-08-25T04:40:18.463 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019d2c0 con 0x7fb9f0101450 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9f48c7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019d450 con 0x7fb9f00ff8b0 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9eed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9eed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33142/0 (socket says 172.21.15.162:33142) 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9eed9d700 1 -- 172.21.15.162:0/2148651258 learned_addr learned my addr 172.21.15.162:0/2148651258 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.459+0000 7fb9ee59c700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 0x7fb9f01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9edd9b700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:18.464 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 -- 172.21.15.162:0/2148651258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 msgr2=0x7fb9f01955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 0x7fb9f01955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 -- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 msgr2=0x7fb9f0195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 -- 172.21.15.162:0/2148651258 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00095f0 con 0x7fb9f0101450 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9edd9b700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9eed9d700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b970 tx=0x7fb9e800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800c930 con 0x7fb9f0101450 2023-08-25T04:40:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f0068440 con 0x7fb9f0101450 2023-08-25T04:40:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9e8014440 con 0x7fb9f0101450 2023-08-25T04:40:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80075d0 con 0x7fb9f0101450 2023-08-25T04:40:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.460+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f00688f0 con 0x7fb9f0101450 2023-08-25T04:40:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.461+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f010b130 con 0x7fb9f0101450 2023-08-25T04:40:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.462+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fb9e80148d0 con 0x7fb9f0101450 2023-08-25T04:40:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.462+0000 7fb9df7fe700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 0x7fb9d80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.463+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fb9e8006e40 con 0x7fb9f0101450 2023-08-25T04:40:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.463+0000 7fb9ee59c700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 0x7fb9d80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.463+0000 7fb9ee59c700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 0x7fb9d80a2410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0100910 tx=0x7fb9e400d620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:40:18.468 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.467+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb9e8094610 con 0x7fb9f0101450 2023-08-25T04:40:18.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:18.663+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9f0106e30 con 0x7fb9d809ff50 2023-08-25T04:40:19.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:40:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:40:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi145 2023-08-25T04:40:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 5755M 2023-08-25T04:40:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] boot 2023-08-25T04:40:19.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T04:40:19.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:19.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:19.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:19.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi145 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 5755M 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] boot 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T04:40:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi145 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 5755M 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: osd.6 [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] boot 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: osdmap e46: 7 total, 7 up, 7 in 2023-08-25T04:40:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:40:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:19 smithi145 ceph-mon[97681]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:20.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:19 smithi145 ceph-mon[97681]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T04:40:20.153 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:19 smithi162 ceph-mon[130941]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:20.153 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:19 smithi162 ceph-mon[130941]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T04:40:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:19 smithi183 ceph-mon[131414]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:19 smithi183 ceph-mon[131414]: osdmap e47: 7 total, 7 up, 7 in 2023-08-25T04:40:21.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:20 smithi145 ceph-mon[97681]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:21.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:20 smithi162 ceph-mon[130941]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:21.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:20 smithi183 ceph-mon[131414]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/699142895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]': finished 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/513435431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/699142895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]': finished 2023-08-25T04:40:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T04:40:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/513435431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/699142895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]: dispatch 2023-08-25T04:40:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94da753b-5dbe-4a36-90c9-6d8d68d9d70b"}]': finished 2023-08-25T04:40:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: osdmap e48: 8 total, 7 up, 8 in 2023-08-25T04:40:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/513435431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:23.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:22 smithi162 ceph-mon[130941]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:22 smithi183 ceph-mon[131414]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:23.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:22 smithi145 ceph-mon[97681]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:25.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:24 smithi162 ceph-mon[130941]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:24 smithi183 ceph-mon[131414]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:25.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:24 smithi145 ceph-mon[97681]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:27.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:26 smithi183 ceph-mon[131414]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:27.225 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:26 smithi162 ceph-mon[130941]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:27.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:26 smithi145 ceph-mon[97681]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T04:40:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T04:40:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-25T04:40:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:29.046 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:28 smithi162 ceph-mon[130941]: Deploying daemon osd.7 on smithi162 2023-08-25T04:40:29.046 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:28 smithi162 ceph-mon[130941]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:29.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:28 smithi183 ceph-mon[131414]: Deploying daemon osd.7 on smithi162 2023-08-25T04:40:29.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:28 smithi183 ceph-mon[131414]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:29.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:28 smithi145 ceph-mon[97681]: Deploying daemon osd.7 on smithi162 2023-08-25T04:40:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:28 smithi145 ceph-mon[97681]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:30 smithi145 ceph-mon[97681]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:30 smithi162 ceph-mon[130941]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:30 smithi183 ceph-mon[131414]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:31.866 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:31.866 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:31.866 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:31.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:31.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:31.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:32.712 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:32 smithi162 ceph-mon[130941]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:32.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:32 smithi145 ceph-mon[97681]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:32.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:32 smithi183 ceph-mon[131414]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:33.923 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.922+0000 7fb9df7fe700 1 -- 172.21.15.162:0/2148651258 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb9f0106e30 con 0x7fb9d809ff50 2023-08-25T04:40:33.923 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 7 on host 'smithi162' 2023-08-25T04:40:33.925 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 msgr2=0x7fb9d80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:33.925 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 0x7fb9d80a2410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0100910 tx=0x7fb9e400d620 comp rx=0 tx=0).stop 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 msgr2=0x7fb9f019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b970 tx=0x7fb9e800bc80 comp rx=0 tx=0).stop 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 shutdown_connections 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb9d809ff50 0x7fb9d80a2410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9f0101450 0x7fb9f019cbb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:33.926 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9f00ff8b0 0x7fb9f01955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:33.927 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 --2- 172.21.15.162:0/2148651258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0100ab0 0x7fb9f0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:33.927 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 >> 172.21.15.162:0/2148651258 conn(0x7fb9f00fae30 msgr2=0x7fb9f0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:40:33.927 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 shutdown_connections 2023-08-25T04:40:33.927 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:40:33.925+0000 7fb9f48c7700 1 -- 172.21.15.162:0/2148651258 wait complete. 2023-08-25T04:40:34.361 DEBUG:teuthology.orchestra.run.smithi162:osd.7> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.7.service 2023-08-25T04:40:34.364 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2023-08-25T04:40:34.364 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-25T04:40:34.393 INFO:journalctl@ceph.osd.7.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:40:34.927 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:40:35.038 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:35.038 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.038 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.039 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:35.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:35.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:35.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.138 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.138 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:35.138 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:35.609 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 04:40:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T04:40:35.387+0000 7fdc9f2a3640 -1 osd.7 0 log_to_monitors true 2023-08-25T04:40:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:35 smithi145 ceph-mon[97681]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:35 smithi145 ceph-mon[97681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.108 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:35 smithi162 ceph-mon[130941]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.108 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:35 smithi162 ceph-mon[130941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:35 smithi183 ceph-mon[131414]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:35 smithi183 ceph-mon[131414]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-25T04:40:36.417 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-25T04:40:36.417 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-25T04:40:36.418 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2023-08-25T04:40:36.418 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2023-08-25T04:40:36.418 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109475 s, 958 MB/s 2023-08-25T04:40:36.418 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2023-08-25T04:40:36.882 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2023-08-25T04:40:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T04:40:37.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:37.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:37.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:36 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.161 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T04:40:36.755+0000 7fdc9491a700 -1 osd.7 0 waiting for initial osdmap 2023-08-25T04:40:37.162 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T04:40:36.766+0000 7fdc8ef0c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:40:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T04:40:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T04:40:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:37.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:37.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:36 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: osdmap e49: 8 total, 7 up, 8 in 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:37.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:36 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:37.433 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:40:37.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi162 2023-08-25T04:40:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 5722M 2023-08-25T04:40:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:40:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T04:40:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:37.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:37.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi162 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 5722M 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi162 2023-08-25T04:40:38.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 5722M 2023-08-25T04:40:38.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:40:38.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: osdmap e50: 8 total, 7 up, 8 in 2023-08-25T04:40:38.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.168 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:38.640 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 -- 172.21.15.183:0/1505499322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94100ac0 msgr2=0x7f0c94100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:38.640 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1505499322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94100ac0 0x7f0c94100f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009990 tx=0x7f0c80009ca0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.640 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 -- 172.21.15.183:0/1505499322 shutdown_connections 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1505499322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94101460 0x7f0c94107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1505499322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c940ff8c0 0x7f0c940ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1505499322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94100ac0 0x7f0c94100f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 -- 172.21.15.183:0/1505499322 >> 172.21.15.183:0/1505499322 conn(0x7f0c940fae60 msgr2=0x7f0c940fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 -- 172.21.15.183:0/1505499322 shutdown_connections 2023-08-25T04:40:38.641 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.637+0000 7f0c9afca700 1 -- 172.21.15.183:0/1505499322 wait complete. 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 Processor -- start 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 -- start start 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c940ff8c0 0x7f0c94195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94100ac0 0x7f0c94195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:38.642 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419d0f0 con 0x7f0c940ff8c0 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419d230 con 0x7f0c94100ac0 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c9afca700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419d3c0 con 0x7f0c94101460 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52342/0 (socket says 172.21.15.183:52342) 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 -- 172.21.15.183:0/1044079636 learned_addr learned my addr 172.21.15.183:0/1044079636 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c93fff700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94100ac0 0x7f0c94195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:38.643 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 -- 172.21.15.183:0/1044079636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94100ac0 msgr2=0x7f0c94195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:38.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94100ac0 0x7f0c94195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 -- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c940ff8c0 msgr2=0x7f0c94195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:38.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c940ff8c0 0x7f0c94195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:38.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 -- 172.21.15.183:0/1044079636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800095f0 con 0x7f0c94101460 2023-08-25T04:40:38.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c99567700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007f10 tx=0x7f0c8c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.638+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c017070 con 0x7f0c94101460 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.639+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c8c00f040 con 0x7f0c94101460 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.639+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9419d6a0 con 0x7f0c94101460 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.639+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c005740 con 0x7f0c94101460 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.639+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9419db50 con 0x7f0c94101460 2023-08-25T04:40:38.645 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.640+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9410b0c0 con 0x7f0c94101460 2023-08-25T04:40:38.646 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.640+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f0c8c0058a0 con 0x7f0c94101460 2023-08-25T04:40:38.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.641+0000 7f0c91ffb700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 0x7f0c840a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:40:38.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.641+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f0c8c0c9ed0 con 0x7f0c94101460 2023-08-25T04:40:38.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.641+0000 7f0c98d66700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 0x7f0c840a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:40:38.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.642+0000 7f0c98d66700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 0x7f0c840a2410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c94100920 tx=0x7f0c88005ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:40:38.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.645+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0c8c0947a0 con 0x7f0c94101460 2023-08-25T04:40:38.844 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:38.844+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c94060d60 con 0x7f0c8409ff50 2023-08-25T04:40:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:40:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:40:39.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:39.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] boot 2023-08-25T04:40:39.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T04:40:39.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:40:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:40:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] boot 2023-08-25T04:40:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T04:40:39.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:40:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:40:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-08-25T04:40:39.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: osd.7 [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] boot 2023-08-25T04:40:39.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: osdmap e51: 8 total, 8 up, 8 in 2023-08-25T04:40:39.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:40:39.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:39 smithi183 ceph-mon[131414]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T04:40:39.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:39 smithi183 ceph-mon[131414]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:39.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:39.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:39.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:39 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:39 smithi145 ceph-mon[97681]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T04:40:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:39 smithi145 ceph-mon[97681]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:40.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:39 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:39 smithi162 ceph-mon[130941]: osdmap e52: 8 total, 8 up, 8 in 2023-08-25T04:40:40.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:39 smithi162 ceph-mon[130941]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:40:40.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:40:40.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:40:40.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:39 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:41.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:40 smithi145 ceph-mon[97681]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:41.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:40 smithi145 ceph-mon[97681]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T04:40:41.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:40 smithi162 ceph-mon[130941]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:41.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:40 smithi162 ceph-mon[130941]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T04:40:41.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:40 smithi183 ceph-mon[131414]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:41.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:40 smithi183 ceph-mon[131414]: osdmap e53: 8 total, 8 up, 8 in 2023-08-25T04:40:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T04:40:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/3206713274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]': finished 2023-08-25T04:40:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T04:40:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:42.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T04:40:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/3206713274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]': finished 2023-08-25T04:40:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T04:40:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: osdmap e54: 8 total, 8 up, 8 in 2023-08-25T04:40:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/3206713274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]: dispatch 2023-08-25T04:40:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "577926f0-d4e5-4c92-9817-38ba81609f92"}]': finished 2023-08-25T04:40:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: osdmap e55: 9 total, 8 up, 9 in 2023-08-25T04:40:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:43.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:42 smithi145 ceph-mon[97681]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:43.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:42 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/3268887996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:43.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:42 smithi162 ceph-mon[130941]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:42 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/3268887996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:43.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:42 smithi183 ceph-mon[131414]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:43.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:42 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/3268887996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:40:45.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:44 smithi145 ceph-mon[97681]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:45.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:44 smithi162 ceph-mon[130941]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:45.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:44 smithi183 ceph-mon[131414]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:47.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:46 smithi145 ceph-mon[97681]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:47.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:46 smithi162 ceph-mon[130941]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:47.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:46 smithi183 ceph-mon[131414]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:48.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:47 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T04:40:48.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:47 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:48.158 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:47 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T04:40:48.158 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:47 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:47 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-25T04:40:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:47 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:49.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:48 smithi145 ceph-mon[97681]: Deploying daemon osd.8 on smithi183 2023-08-25T04:40:49.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:48 smithi145 ceph-mon[97681]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:49.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:48 smithi162 ceph-mon[130941]: Deploying daemon osd.8 on smithi183 2023-08-25T04:40:49.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:48 smithi162 ceph-mon[130941]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:49.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:48 smithi183 ceph-mon[131414]: Deploying daemon osd.8 on smithi183 2023-08-25T04:40:49.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:48 smithi183 ceph-mon[131414]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:50.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:50 smithi145 ceph-mon[97681]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:50.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:50 smithi183 ceph-mon[131414]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:50.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:50 smithi162 ceph-mon[130941]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:52.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:52 smithi183 ceph-mon[131414]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:52.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:52.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:52.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:53.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:52 smithi162 ceph-mon[130941]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:53.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:52 smithi145 ceph-mon[97681]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:53.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:53.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:53.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:54.560 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.558+0000 7f0c91ffb700 1 -- 172.21.15.183:0/1044079636 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0c94060d60 con 0x7f0c8409ff50 2023-08-25T04:40:54.560 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2023-08-25T04:40:54.562 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 msgr2=0x7f0c840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:54.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 0x7f0c840a2410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c94100920 tx=0x7f0c88005ca0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 msgr2=0x7f0c9419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:40:54.563 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007f10 tx=0x7f0c8c00d3b0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 shutdown_connections 2023-08-25T04:40:54.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0c8409ff50 0x7f0c840a2410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.564 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c94101460 0x7f0c9419cb20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.565 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c940ff8c0 0x7f0c94195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.565 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 --2- 172.21.15.183:0/1044079636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c94100ac0 0x7f0c94195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:40:54.565 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 >> 172.21.15.183:0/1044079636 conn(0x7f0c940fae60 msgr2=0x7f0c940fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:40:54.565 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 shutdown_connections 2023-08-25T04:40:54.566 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:40:54.560+0000 7f0c9afca700 1 -- 172.21.15.183:0/1044079636 wait complete. 2023-08-25T04:40:54.927 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.8.service 2023-08-25T04:40:54.929 INFO:tasks.cephadm:Deploying osd.9 on smithi145 with /dev/vg_nvme/lv_1... 2023-08-25T04:40:54.930 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T04:40:54.958 INFO:journalctl@ceph.osd.8.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:40:55.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:55.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:55.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:55.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:55.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:55.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:40:55.656 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:40:56.173 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 04:40:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T04:40:55.823+0000 7ff2a7915640 -1 osd.8 0 log_to_monitors true 2023-08-25T04:40:56.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:55 smithi183 ceph-mon[131414]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:56.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:55 smithi183 ceph-mon[131414]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:56.239 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:55 smithi145 ceph-mon[97681]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:56.239 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:55 smithi145 ceph-mon[97681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:56.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:55 smithi162 ceph-mon[130941]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:56.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:55 smithi162 ceph-mon[130941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-25T04:40:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T04:40:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T04:40:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:56 smithi145 ceph-mon[97681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.153 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T04:40:56.927+0000 7ff29cf8c700 -1 osd.8 0 waiting for initial osdmap 2023-08-25T04:40:57.153 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T04:40:56.941+0000 7ff294d79700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:56 smithi183 ceph-mon[131414]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:57.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-25T04:40:57.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: osdmap e56: 9 total, 8 up, 9 in 2023-08-25T04:40:57.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:57.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: from='osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:56 smithi162 ceph-mon[130941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:40:57.499 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T04:40:57.500 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T04:40:57.500 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2023-08-25T04:40:57.500 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2023-08-25T04:40:57.501 INFO:teuthology.orchestra.run.smithi145.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0421357 s, 249 MB/s 2023-08-25T04:40:57.501 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 2023-08-25T04:40:57.501 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2023-08-25T04:40:57.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2023-08-25T04:40:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:40:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T04:40:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:58.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:58.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] boot 2023-08-25T04:40:58.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T04:40:58.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:40:58.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T04:40:58.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:58.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:58.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] boot 2023-08-25T04:40:58.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T04:40:58.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: osdmap e57: 9 total, 8 up, 9 in 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:40:58.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:40:58.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:40:58.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:40:58.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: osd.8 [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] boot 2023-08-25T04:40:58.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: osdmap e58: 9 total, 9 up, 9 in 2023-08-25T04:40:58.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:40:58.658 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:40:59.177 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:58 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:40:59.177 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:58 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:40:59.177 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:58 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi183 2023-08-25T04:40:59.177 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:58 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 5722M 2023-08-25T04:40:59.178 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:58 smithi145 ceph-mon[97681]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:59.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:58 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:40:59.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:58 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:40:59.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:58 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi183 2023-08-25T04:40:59.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:58 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 5722M 2023-08-25T04:40:59.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:58 smithi162 ceph-mon[130941]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:40:59.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:58 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:40:59.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:58 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:40:59.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:58 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi183 2023-08-25T04:40:59.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:58 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 5722M 2023-08-25T04:40:59.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:58 smithi183 ceph-mon[131414]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-08-25T04:41:00.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 -- 172.21.15.145:0/1161121647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28101bb0 msgr2=0x7ffb28102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:00.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 --2- 172.21.15.145:0/1161121647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28101bb0 0x7ffb28102010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb180099c0 tx=0x7ffb18009cd0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 -- 172.21.15.145:0/1161121647 shutdown_connections 2023-08-25T04:41:00.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 --2- 172.21.15.145:0/1161121647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb28102550 0x7ffb28108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 --2- 172.21.15.145:0/1161121647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb281009b0 0x7ffb28100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 --2- 172.21.15.145:0/1161121647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28101bb0 0x7ffb28102010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 -- 172.21.15.145:0/1161121647 >> 172.21.15.145:0/1161121647 conn(0x7ffb280fbf10 msgr2=0x7ffb280fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:00.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 -- 172.21.15.145:0/1161121647 shutdown_connections 2023-08-25T04:41:00.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.031+0000 7ffb303ec700 1 -- 172.21.15.145:0/1161121647 wait complete. 2023-08-25T04:41:00.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.032+0000 7ffb303ec700 1 Processor -- start 2023-08-25T04:41:00.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.032+0000 7ffb303ec700 1 -- start start 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb28101bb0 0x7ffb28198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28102550 0x7ffb2819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281a0440 con 0x7ffb28102550 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281a0580 con 0x7ffb281009b0 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb303ec700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281a0710 con 0x7ffb28101bb0 2023-08-25T04:41:00.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:53168/0 (socket says 172.21.15.145:53168) 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 -- 172.21.15.145:0/2982027440 learned_addr learned my addr 172.21.15.145:0/2982027440 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e989700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28102550 0x7ffb2819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2d987700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb28101bb0 0x7ffb28198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 -- 172.21.15.145:0/2982027440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb28101bb0 msgr2=0x7ffb28198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb28101bb0 0x7ffb28198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 -- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28102550 msgr2=0x7ffb2819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:00.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28102550 0x7ffb2819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:00.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.033+0000 7ffb2e188700 1 -- 172.21.15.145:0/2982027440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180095f0 con 0x7ffb281009b0 2023-08-25T04:41:00.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb2e188700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400d980 tx=0x7ffb2400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24004020 con 0x7ffb281009b0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb281a09f0 con 0x7ffb281009b0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb2400f690 con 0x7ffb281009b0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24003a80 con 0x7ffb281009b0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.034+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb281a0f40 con 0x7ffb281009b0 2023-08-25T04:41:00.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.035+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28192ff0 con 0x7ffb281009b0 2023-08-25T04:41:00.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.037+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ffb2400f800 con 0x7ffb281009b0 2023-08-25T04:41:00.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.038+0000 7ffb1f7fe700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 0x7ffb140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:00.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.038+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7ffb240ca5a0 con 0x7ffb281009b0 2023-08-25T04:41:00.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.039+0000 7ffb2d987700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 0x7ffb140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:00.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.039+0000 7ffb2d987700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 0x7ffb140a2490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b580 tx=0x7ffb18005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:00.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.041+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ffb24094ca0 con 0x7ffb281009b0 2023-08-25T04:41:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:40:59 smithi145 ceph-mon[97681]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T04:41:00.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:00.249+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb2802ce10 con 0x7ffb1409ffd0 2023-08-25T04:41:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:40:59 smithi162 ceph-mon[130941]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T04:41:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:40:59 smithi183 ceph-mon[131414]: osdmap e59: 9 total, 9 up, 9 in 2023-08-25T04:41:01.288 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:00 smithi145 ceph-mon[97681]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:01.288 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:00 smithi145 ceph-mon[97681]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T04:41:01.289 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:01.289 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:01.289 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:01.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:00 smithi162 ceph-mon[130941]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:01.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:00 smithi162 ceph-mon[130941]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T04:41:01.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:01.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:01.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:01.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:00 smithi183 ceph-mon[131414]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:01.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:00 smithi183 ceph-mon[131414]: osdmap e60: 9 total, 9 up, 9 in 2023-08-25T04:41:01.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:01.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:01.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:01 smithi145 ceph-mon[97681]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:01 smithi145 ceph-mon[97681]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T04:41:02.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:01 smithi162 ceph-mon[130941]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:02.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:01 smithi162 ceph-mon[130941]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T04:41:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:01 smithi183 ceph-mon[131414]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:01 smithi183 ceph-mon[131414]: osdmap e61: 9 total, 9 up, 9 in 2023-08-25T04:41:03.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:02 smithi145 ceph-mon[97681]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:03.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:02 smithi145 ceph-mon[97681]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T04:41:03.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:02 smithi162 ceph-mon[130941]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:03.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:02 smithi162 ceph-mon[130941]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T04:41:03.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:02 smithi183 ceph-mon[131414]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:03.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:02 smithi183 ceph-mon[131414]: osdmap e62: 9 total, 9 up, 9 in 2023-08-25T04:41:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:03 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]: dispatch 2023-08-25T04:41:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:03 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]': finished 2023-08-25T04:41:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:03 smithi145 ceph-mon[97681]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T04:41:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:04.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:03 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3622904764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:04.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:03 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]: dispatch 2023-08-25T04:41:04.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:03 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]': finished 2023-08-25T04:41:04.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:03 smithi162 ceph-mon[130941]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T04:41:04.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:04.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:03 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3622904764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:03 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]: dispatch 2023-08-25T04:41:04.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:03 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2325133032' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fc38b1d-c461-49db-a3cc-64ba6780cf5d"}]': finished 2023-08-25T04:41:04.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:03 smithi183 ceph-mon[131414]: osdmap e63: 10 total, 9 up, 10 in 2023-08-25T04:41:04.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:04.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:03 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3622904764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:05.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:05 smithi145 ceph-mon[97681]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:05.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:05 smithi162 ceph-mon[130941]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:05.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:05 smithi183 ceph-mon[131414]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:07.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:07 smithi145 ceph-mon[97681]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-08-25T04:41:07.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:07 smithi162 ceph-mon[130941]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-08-25T04:41:07.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:07 smithi183 ceph-mon[131414]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-08-25T04:41:09.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:09 smithi145 ceph-mon[97681]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-08-25T04:41:09.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:09 smithi162 ceph-mon[130941]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-08-25T04:41:09.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:09 smithi183 ceph-mon[131414]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-08-25T04:41:10.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:10 smithi145 ceph-mon[97681]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:10 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T04:41:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:10 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:10.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:10 smithi162 ceph-mon[130941]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:10.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:10 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T04:41:10.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:10 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:10.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:10 smithi183 ceph-mon[131414]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:10 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-25T04:41:10.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:10 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:11.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:11 smithi145 ceph-mon[97681]: Deploying daemon osd.9 on smithi145 2023-08-25T04:41:11.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:11 smithi162 ceph-mon[130941]: Deploying daemon osd.9 on smithi145 2023-08-25T04:41:11.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:11 smithi183 ceph-mon[131414]: Deploying daemon osd.9 on smithi145 2023-08-25T04:41:12.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:12 smithi145 ceph-mon[97681]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-08-25T04:41:12.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:12 smithi162 ceph-mon[130941]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-08-25T04:41:12.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:12 smithi183 ceph-mon[131414]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-08-25T04:41:14.985 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:14 smithi145 ceph-mon[97681]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-25T04:41:15.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:14 smithi162 ceph-mon[130941]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-25T04:41:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:14 smithi183 ceph-mon[131414]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-08-25T04:41:17.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:16 smithi145 ceph-mon[97681]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:41:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:16 smithi162 ceph-mon[130941]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:41:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:16 smithi183 ceph-mon[131414]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:41:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:18.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:18 smithi145 ceph-mon[97681]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:18 smithi162 ceph-mon[130941]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:18 smithi183 ceph-mon[131414]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:19.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.503+0000 7ffb1f7fe700 1 -- 172.21.15.145:0/2982027440 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb2802ce10 con 0x7ffb1409ffd0 2023-08-25T04:41:19.504 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 9 on host 'smithi145' 2023-08-25T04:41:19.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 msgr2=0x7ffb140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:19.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 0x7ffb140a2490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b580 tx=0x7ffb18005fb0 comp rx=0 tx=0).stop 2023-08-25T04:41:19.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 msgr2=0x7ffb281988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400d980 tx=0x7ffb2400dc90 comp rx=0 tx=0).stop 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 shutdown_connections 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb1409ffd0 0x7ffb140a2490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb28102550 0x7ffb2819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb281009b0 0x7ffb281988b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 --2- 172.21.15.145:0/2982027440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb28101bb0 0x7ffb28198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:19.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.507+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 >> 172.21.15.145:0/2982027440 conn(0x7ffb280fbf10 msgr2=0x7ffb281074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:19.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.508+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 shutdown_connections 2023-08-25T04:41:19.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:41:19.508+0000 7ffb303ec700 1 -- 172.21.15.145:0/2982027440 wait complete. 2023-08-25T04:41:19.885 DEBUG:teuthology.orchestra.run.smithi145:osd.9> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.9.service 2023-08-25T04:41:19.890 INFO:tasks.cephadm:Deploying osd.10 on smithi162 with /dev/vg_nvme/lv_1... 2023-08-25T04:41:19.890 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T04:41:19.965 INFO:journalctl@ceph.osd.9.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:41:20.455 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:41:20.463 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T04:41:20.171+0000 7fb95ce60640 -1 osd.9 0 log_to_monitors true 2023-08-25T04:41:20.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.780 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:20.781 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:20 smithi145 ceph-mon[97681]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T04:41:20.814 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.815 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:20.816 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:20 smithi162 ceph-mon[130941]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T04:41:20.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:20.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:20 smithi183 ceph-mon[131414]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-25T04:41:21.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:21 smithi162 ceph-mon[130941]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T04:41:21.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:21 smithi162 ceph-mon[130941]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T04:41:21.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:21.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:21 smithi162 ceph-mon[130941]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:41:21.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T04:41:21.577+0000 7fb9524d7700 -1 osd.9 0 waiting for initial osdmap 2023-08-25T04:41:21.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T04:41:21.590+0000 7fb94bac7700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:41:21.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-mon[97681]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T04:41:21.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-mon[97681]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T04:41:21.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:21.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:21 smithi145 ceph-mon[97681]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:41:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T04:41:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T04:41:21.840 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2023-08-25T04:41:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2023-08-25T04:41:21.841 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149652 s, 701 MB/s 2023-08-25T04:41:21.841 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2023-08-25T04:41:21.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:21 smithi183 ceph-mon[131414]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-25T04:41:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:21 smithi183 ceph-mon[131414]: osdmap e64: 10 total, 9 up, 10 in 2023-08-25T04:41:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:21 smithi183 ceph-mon[131414]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2023-08-25T04:41:22.300 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi162:vg_nvme/lv_1 2023-08-25T04:41:22.853 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T04:41:22.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:22 smithi162 ceph-mon[130941]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:41:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:22 smithi162 ceph-mon[130941]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T04:41:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:22.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:22 smithi162 ceph-mon[130941]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:22.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:22 smithi183 ceph-mon[131414]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:41:22.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:22 smithi183 ceph-mon[131414]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T04:41:22.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:22.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:22.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:22 smithi183 ceph-mon[131414]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:23.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:22 smithi145 ceph-mon[97681]: from='osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2023-08-25T04:41:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:22 smithi145 ceph-mon[97681]: osdmap e65: 10 total, 9 up, 10 in 2023-08-25T04:41:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:22 smithi145 ceph-mon[97681]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-08-25T04:41:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:41:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:41:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] boot 2023-08-25T04:41:23.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi145 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 4316M 2023-08-25T04:41:23.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:23.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:23.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:23 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:41:23.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] boot 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi145 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 4316M 2023-08-25T04:41:23.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:23.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:23.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:23.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:23.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:23.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:23 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 -- 172.21.15.162:0/916441142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c0ffac0 msgr2=0x7f436c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 --2- 172.21.15.162:0/916441142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c0ffac0 0x7f436c0fff40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4368009990 tx=0x7f4368009ca0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 -- 172.21.15.162:0/916441142 shutdown_connections 2023-08-25T04:41:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 --2- 172.21.15.162:0/916441142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c100480 0x7f436c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 --2- 172.21.15.162:0/916441142 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100e60 0x7f436c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 --2- 172.21.15.162:0/916441142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c0ffac0 0x7f436c0fff40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.933+0000 7f4373344700 1 -- 172.21.15.162:0/916441142 >> 172.21.15.162:0/916441142 conn(0x7f436c0fa5d0 msgr2=0x7f436c0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:23.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- 172.21.15.162:0/916441142 shutdown_connections 2023-08-25T04:41:23.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- 172.21.15.162:0/916441142 wait complete. 2023-08-25T04:41:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 Processor -- start 2023-08-25T04:41:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- start start 2023-08-25T04:41:23.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0ffac0 0x7f436c195520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 0x7f436c195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c19d0b0 con 0x7f436c100480 2023-08-25T04:41:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c19d1f0 con 0x7f436c100e60 2023-08-25T04:41:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f4373344700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c19d380 con 0x7f436c0ffac0 2023-08-25T04:41:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f43718e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f43718e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:33286/0 (socket says 172.21.15.162:33286) 2023-08-25T04:41:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f43718e1700 1 -- 172.21.15.162:0/1852169337 learned_addr learned my addr 172.21.15.162:0/1852169337 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-08-25T04:41:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.934+0000 7f43710e0700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0ffac0 0x7f436c195520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43708df700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 0x7f436c195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:23.944 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 -- 172.21.15.162:0/1852169337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0ffac0 msgr2=0x7f436c195520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:23.944 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0ffac0 0x7f436c195520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.944 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 -- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 msgr2=0x7f436c195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:23.944 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 0x7f436c195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:23.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 -- 172.21.15.162:0/1852169337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680095f0 con 0x7f436c100e60 2023-08-25T04:41:23.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43708df700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 0x7f436c195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:41:23.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43718e1700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4364007d70 tx=0x7f436400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:23.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436400ad40 con 0x7f436c100e60 2023-08-25T04:41:23.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43640128e0 con 0x7f436c100e60 2023-08-25T04:41:23.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c19d660 con 0x7f436c100e60 2023-08-25T04:41:23.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640116b0 con 0x7f436c100e60 2023-08-25T04:41:23.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.935+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c19db10 con 0x7f436c100e60 2023-08-25T04:41:23.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.936+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c106ed0 con 0x7f436c100e60 2023-08-25T04:41:23.948 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.937+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4364012a50 con 0x7f436c100e60 2023-08-25T04:41:23.948 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.937+0000 7f43627fc700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 0x7f43580a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:23.949 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.938+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f43640cac70 con 0x7f436c100e60 2023-08-25T04:41:23.949 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.938+0000 7f43710e0700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 0x7f43580a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:23.949 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.938+0000 7f43710e0700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 0x7f43580a2410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f435c005950 tx=0x7f435c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:23.949 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:23.942+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4364095200 con 0x7f436c100e60 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: osd.9 [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] boot 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: osdmap e66: 10 total, 10 up, 10 in 2023-08-25T04:41:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi145 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 4316M 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:24.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:24.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:24.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:24.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:23 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:24.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:24.138+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f436c02ce30 con 0x7f435809ff50 2023-08-25T04:41:25.207 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:24 smithi162 ceph-mon[130941]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:25.207 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:24 smithi162 ceph-mon[130941]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T04:41:25.208 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:25.208 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:25.208 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:25.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:24 smithi145 ceph-mon[97681]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:24 smithi145 ceph-mon[97681]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T04:41:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:25.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:25.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:24 smithi183 ceph-mon[131414]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:25.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:24 smithi183 ceph-mon[131414]: osdmap e67: 10 total, 10 up, 10 in 2023-08-25T04:41:25.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:25.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:25.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:26.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:25 smithi145 ceph-mon[97681]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:26.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:25 smithi162 ceph-mon[130941]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:26.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:25 smithi183 ceph-mon[131414]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/1143131572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]': finished 2023-08-25T04:41:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T04:41:27.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/1143131572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]': finished 2023-08-25T04:41:27.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T04:41:27.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:27.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/1143131572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]: dispatch 2023-08-25T04:41:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0aa5c31-938d-470f-bbba-9d2237cddae4"}]': finished 2023-08-25T04:41:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: osdmap e68: 11 total, 10 up, 11 in 2023-08-25T04:41:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:27 smithi145 ceph-mon[97681]: from='client.? 172.21.15.162:0/3212746888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:28.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:27 smithi162 ceph-mon[130941]: from='client.? 172.21.15.162:0/3212746888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:28.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:27 smithi183 ceph-mon[131414]: from='client.? 172.21.15.162:0/3212746888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:28 smithi145 ceph-mon[97681]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:29.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:28 smithi162 ceph-mon[130941]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:29.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:28 smithi183 ceph-mon[131414]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:30.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:30 smithi162 ceph-mon[130941]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:30.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:30 smithi145 ceph-mon[97681]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:30 smithi183 ceph-mon[131414]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:33.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:32 smithi145 ceph-mon[97681]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:32 smithi162 ceph-mon[130941]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:33.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:32 smithi183 ceph-mon[131414]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:33.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T04:41:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:33 smithi162 ceph-mon[130941]: Deploying daemon osd.10 on smithi162 2023-08-25T04:41:34.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T04:41:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:33 smithi145 ceph-mon[97681]: Deploying daemon osd.10 on smithi162 2023-08-25T04:41:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-25T04:41:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:33 smithi183 ceph-mon[131414]: Deploying daemon osd.10 on smithi162 2023-08-25T04:41:35.002 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:34 smithi162 ceph-mon[130941]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:35.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:34 smithi145 ceph-mon[97681]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:35.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:34 smithi183 ceph-mon[131414]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:36.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:36 smithi162 ceph-mon[130941]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:37.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:36 smithi145 ceph-mon[97681]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:36 smithi183 ceph-mon[131414]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:37.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:37.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:37.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:37 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:37 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:38.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:38.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:37 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:38 smithi145 ceph-mon[97681]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:39.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:38 smithi162 ceph-mon[130941]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:38 smithi183 ceph-mon[131414]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:39.200 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 10 on host 'smithi162' 2023-08-25T04:41:39.200 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.199+0000 7f43627fc700 1 -- 172.21.15.162:0/1852169337 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f436c02ce30 con 0x7f435809ff50 2023-08-25T04:41:39.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 msgr2=0x7f43580a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:39.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 0x7f43580a2410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f435c005950 tx=0x7f435c00b410 comp rx=0 tx=0).stop 2023-08-25T04:41:39.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 msgr2=0x7f436c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4364007d70 tx=0x7f436400a340 comp rx=0 tx=0).stop 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 shutdown_connections 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f435809ff50 0x7f43580a2410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f436c100e60 0x7f436c19cae0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c0ffac0 0x7f436c195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 --2- 172.21.15.162:0/1852169337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f436c100480 0x7f436c195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:39.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.201+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 >> 172.21.15.162:0/1852169337 conn(0x7f436c0fa5d0 msgr2=0x7f436c104010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:39.204 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.202+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 shutdown_connections 2023-08-25T04:41:39.204 INFO:teuthology.orchestra.run.smithi162.stderr:2023-08-25T04:41:39.202+0000 7f4373344700 1 -- 172.21.15.162:0/1852169337 wait complete. 2023-08-25T04:41:39.650 DEBUG:teuthology.orchestra.run.smithi162:osd.10> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.10.service 2023-08-25T04:41:39.653 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2023-08-25T04:41:39.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-25T04:41:39.681 INFO:journalctl@ceph.osd.10.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:41:40.215 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:41:40.337 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.337 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.338 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.338 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.338 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-mon[130941]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:40.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:40 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:40 smithi183 ceph-mon[131414]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:40 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:40 smithi145 ceph-mon[97681]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:41.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 04:41:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T04:41:40.795+0000 7f5d32c6e640 -1 osd.10 0 log_to_monitors true 2023-08-25T04:41:41.617 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:41 smithi162 ceph-mon[130941]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:41.618 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:41 smithi162 ceph-mon[130941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:41.672 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-25T04:41:41.672 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-25T04:41:41.673 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2023-08-25T04:41:41.673 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2023-08-25T04:41:41.673 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114667 s, 914 MB/s 2023-08-25T04:41:41.673 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2023-08-25T04:41:41.817 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:41 smithi183 ceph-mon[131414]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:41.818 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:41 smithi183 ceph-mon[131414]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:41.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:41 smithi145 ceph-mon[97681]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:41.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:41 smithi145 ceph-mon[97681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-25T04:41:42.071 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2023-08-25T04:41:42.623 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:41:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T04:41:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T04:41:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi162 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 4292M 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:42.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:42 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T04:41:42.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi162 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 4292M 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:42.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:42 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.911 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T04:41:42.531+0000 7f5d282e5700 -1 osd.10 0 waiting for initial osdmap 2023-08-25T04:41:42.912 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T04:41:42.543+0000 7f5d218d5700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:41:42.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: osdmap e69: 11 total, 10 up, 11 in 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi162 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:41:42.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 4292M 2023-08-25T04:41:42.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:42.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:42.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:42.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:42.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:41:42.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:42 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:43.787 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 -- 172.21.15.183:0/3033788998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc100ab0 msgr2=0x7f75fc100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:43.787 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 --2- 172.21.15.183:0/3033788998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc100ab0 0x7f75fc100f10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009990 tx=0x7f75e4009ca0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 -- 172.21.15.183:0/3033788998 shutdown_connections 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 --2- 172.21.15.183:0/3033788998 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc101450 0x7f75fc107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 --2- 172.21.15.183:0/3033788998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 0x7f75fc0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 --2- 172.21.15.183:0/3033788998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc100ab0 0x7f75fc100f10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 -- 172.21.15.183:0/3033788998 >> 172.21.15.183:0/3033788998 conn(0x7f75fc0fae30 msgr2=0x7f75fc0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:43.788 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 -- 172.21.15.183:0/3033788998 shutdown_connections 2023-08-25T04:41:43.789 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 -- 172.21.15.183:0/3033788998 wait complete. 2023-08-25T04:41:43.790 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.786+0000 7f76011cd700 1 Processor -- start 2023-08-25T04:41:43.790 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 -- start start 2023-08-25T04:41:43.790 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 0x7f75fc103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:43.790 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc100ab0 0x7f75fc101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc102b50 con 0x7f75fc100ab0 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc102cc0 con 0x7f75fc0ff8b0 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f76011cd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc1a0780 con 0x7f75fc101450 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52476/0 (socket says 172.21.15.183:52476) 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 -- 172.21.15.183:0/1785884296 learned_addr learned my addr 172.21.15.183:0/1785884296 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:41:43.791 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fad9d700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 0x7f75fc103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:43.792 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fa59c700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc100ab0 0x7f75fc101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:43.792 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 -- 172.21.15.183:0/1785884296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 msgr2=0x7f75fc103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 0x7f75fc103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 -- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc100ab0 msgr2=0x7f75fc101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc100ab0 0x7f75fc101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 -- 172.21.15.183:0/1785884296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40095f0 con 0x7f75fc101450 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.787+0000 7f75fb59e700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c970 tx=0x7f75ec00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:43.793 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.788+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec004a30 con 0x7f75fc101450 2023-08-25T04:41:43.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.788+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc1a0a60 con 0x7f75fc101450 2023-08-25T04:41:43.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.788+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75ec0054e0 con 0x7f75fc101450 2023-08-25T04:41:43.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.788+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00eb20 con 0x7f75fc101450 2023-08-25T04:41:43.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.788+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc1a0fe0 con 0x7f75fc101450 2023-08-25T04:41:43.794 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.789+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc10b150 con 0x7f75fc101450 2023-08-25T04:41:43.795 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.790+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f75ec004b90 con 0x7f75fc101450 2023-08-25T04:41:43.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.790+0000 7f75f3fff700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 0x7f75e80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:41:43.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.790+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f75ec0caa40 con 0x7f75fc101450 2023-08-25T04:41:43.796 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.791+0000 7f75fad9d700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 0x7f75e80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:41:43.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.791+0000 7f75fad9d700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 0x7f75e80a2410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75fc100910 tx=0x7f75f400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:41:43.797 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.794+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f75ec094e00 con 0x7f75fc101450 2023-08-25T04:41:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:43 smithi145 ceph-mon[97681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:41:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:43 smithi145 ceph-mon[97681]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T04:41:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:43 smithi162 ceph-mon[130941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:41:43.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:43 smithi162 ceph-mon[130941]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T04:41:43.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:43 smithi183 ceph-mon[131414]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2023-08-25T04:41:43.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:43 smithi183 ceph-mon[131414]: osdmap e70: 11 total, 10 up, 11 in 2023-08-25T04:41:43.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:43.993 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:43.992+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f75fc0fbea0 con 0x7f75e809ff50 2023-08-25T04:41:44.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:41:44.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:41:44.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] boot 2023-08-25T04:41:44.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T04:41:44.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:44.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:44.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:44.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:44.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:44.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:44.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:44 smithi183 ceph-mon[131414]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T04:41:44.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:41:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:41:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] boot 2023-08-25T04:41:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T04:41:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:44.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:44.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:44 smithi145 ceph-mon[97681]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T04:41:44.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:41:44.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:41:44.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: osd.10 [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] boot 2023-08-25T04:41:44.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: osdmap e71: 11 total, 11 up, 11 in 2023-08-25T04:41:44.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:41:44.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-08-25T04:41:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:41:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-25T04:41:44.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-25T04:41:44.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:44.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:44 smithi162 ceph-mon[130941]: osdmap e72: 11 total, 11 up, 11 in 2023-08-25T04:41:46.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T04:41:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/3500001853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]': finished 2023-08-25T04:41:46.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: osdmap e74: 12 total, 11 up, 12 in 2023-08-25T04:41:46.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:46.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T04:41:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/3500001853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]': finished 2023-08-25T04:41:46.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: osdmap e74: 12 total, 11 up, 12 in 2023-08-25T04:41:46.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:46.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: osdmap e73: 11 total, 11 up, 11 in 2023-08-25T04:41:46.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:46.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/3500001853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]: dispatch 2023-08-25T04:41:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e"}]': finished 2023-08-25T04:41:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: osdmap e74: 12 total, 11 up, 12 in 2023-08-25T04:41:46.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:47.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:47 smithi145 ceph-mon[97681]: from='client.? 172.21.15.183:0/40469684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:47.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:47 smithi145 ceph-mon[97681]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T04:41:47.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:47 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:47.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:47 smithi162 ceph-mon[130941]: from='client.? 172.21.15.183:0/40469684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:47.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:47 smithi162 ceph-mon[130941]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T04:41:47.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:47 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:47.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:47 smithi183 ceph-mon[131414]: from='client.? 172.21.15.183:0/40469684' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-25T04:41:47.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:47 smithi183 ceph-mon[131414]: osdmap e75: 12 total, 11 up, 12 in 2023-08-25T04:41:47.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:47 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:41:48.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:48 smithi145 ceph-mon[97681]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:48.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:48 smithi162 ceph-mon[130941]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:48.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:48 smithi183 ceph-mon[131414]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:41:50.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:50 smithi145 ceph-mon[97681]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:41:50.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:50 smithi162 ceph-mon[130941]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:41:50.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:50 smithi183 ceph-mon[131414]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-08-25T04:41:52.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:52 smithi183 ceph-mon[131414]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:41:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T04:41:52.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:52 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:53.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:52 smithi145 ceph-mon[97681]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:41:53.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T04:41:53.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:52 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:53.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:52 smithi162 ceph-mon[130941]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-08-25T04:41:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-25T04:41:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:52 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:41:53.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:53 smithi183 ceph-mon[131414]: Deploying daemon osd.11 on smithi183 2023-08-25T04:41:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:53 smithi145 ceph-mon[97681]: Deploying daemon osd.11 on smithi183 2023-08-25T04:41:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:53 smithi162 ceph-mon[130941]: Deploying daemon osd.11 on smithi183 2023-08-25T04:41:55.057 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:54 smithi183 ceph-mon[131414]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:55.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:54 smithi145 ceph-mon[97681]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:55.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:54 smithi162 ceph-mon[130941]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-25T04:41:56.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:56 smithi183 ceph-mon[131414]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T04:41:57.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:56 smithi145 ceph-mon[97681]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T04:41:57.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:56 smithi162 ceph-mon[130941]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-08-25T04:41:58.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:57 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:57 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:58.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:41:58.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:57 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:41:59.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:41:58 smithi145 ceph-mon[97681]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-08-25T04:41:59.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:41:58 smithi162 ceph-mon[130941]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-08-25T04:41:59.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:41:58 smithi183 ceph-mon[131414]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-08-25T04:41:59.252 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2023-08-25T04:41:59.253 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.251+0000 7f75f3fff700 1 -- 172.21.15.183:0/1785884296 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f75fc0fbea0 con 0x7f75e809ff50 2023-08-25T04:41:59.255 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 msgr2=0x7f75e80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:59.255 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 0x7f75e80a2410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75fc100910 tx=0x7f75f400d040 comp rx=0 tx=0).stop 2023-08-25T04:41:59.255 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 msgr2=0x7f75fc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:41:59.255 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c970 tx=0x7f75ec00cc80 comp rx=0 tx=0).stop 2023-08-25T04:41:59.255 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 shutdown_connections 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f75e809ff50 0x7f75e80a2410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75fc101450 0x7f75fc1023d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75fc0ff8b0 0x7f75fc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 --2- 172.21.15.183:0/1785884296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75fc100ab0 0x7f75fc101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 >> 172.21.15.183:0/1785884296 conn(0x7f75fc0fae30 msgr2=0x7f75fc0fb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.253+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 shutdown_connections 2023-08-25T04:41:59.256 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:41:59.254+0000 7f76011cd700 1 -- 172.21.15.183:0/1785884296 wait complete. 2023-08-25T04:41:59.717 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.11.service 2023-08-25T04:41:59.721 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-08-25T04:41:59.721 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd stat -f json 2023-08-25T04:41:59.751 INFO:journalctl@ceph.osd.11.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:42:00.414 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.414 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.414 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.415 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.415 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-mon[131414]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:42:00.453 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:42:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:00 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:00 smithi145 ceph-mon[97681]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:42:00.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:00 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:00.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:00 smithi162 ceph-mon[130941]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-25T04:42:01.172 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 04:42:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T04:42:00.760+0000 7f113ae6e640 -1 osd.11 0 log_to_monitors true 2023-08-25T04:42:01.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:01 smithi183 ceph-mon[131414]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:01.796 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:01 smithi183 ceph-mon[131414]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:01.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:01 smithi145 ceph-mon[97681]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:01.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:01 smithi145 ceph-mon[97681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:01.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 -- 172.21.15.145:0/1162160714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca44101450 msgr2=0x7fca44107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:01.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 --2- 172.21.15.145:0/1162160714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca44101450 0x7fca44107d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00b0d0 tx=0x7fca3c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 -- 172.21.15.145:0/1162160714 shutdown_connections 2023-08-25T04:42:01.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 --2- 172.21.15.145:0/1162160714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca44101450 0x7fca44107d00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 --2- 172.21.15.145:0/1162160714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca440ff8b0 0x7fca440ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 --2- 172.21.15.145:0/1162160714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 -- 172.21.15.145:0/1162160714 >> 172.21.15.145:0/1162160714 conn(0x7fca440fae30 msgr2=0x7fca440fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:01.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 -- 172.21.15.145:0/1162160714 shutdown_connections 2023-08-25T04:42:01.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.834+0000 7fca4a902700 1 -- 172.21.15.145:0/1162160714 wait complete. 2023-08-25T04:42:01.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 Processor -- start 2023-08-25T04:42:01.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 -- start start 2023-08-25T04:42:01.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:01.840 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:01.840 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 0x7fca4419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:01.840 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441a0520 con 0x7fca440ff8b0 2023-08-25T04:42:01.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441a0660 con 0x7fca44101450 2023-08-25T04:42:01.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.836+0000 7fca4a902700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441a07f0 con 0x7fca44100ab0 2023-08-25T04:42:01.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:01.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:01.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60274/0 (socket says 172.21.15.145:60274) 2023-08-25T04:42:01.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 -- 172.21.15.145:0/500727097 learned_addr learned my addr 172.21.15.145:0/500727097 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:01.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca48e9f700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 0x7fca4419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:01.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 -- 172.21.15.145:0/500727097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 msgr2=0x7fca44198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:01.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 -- 172.21.15.145:0/500727097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 msgr2=0x7fca4419ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:01.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 0x7fca4419ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:01.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca43fff700 1 -- 172.21.15.145:0/500727097 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009d00 con 0x7fca440ff8b0 2023-08-25T04:42:01.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca437fe700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:01.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.837+0000 7fca48e9f700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 0x7fca4419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:01.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca43fff700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e9a0 tx=0x7fca3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:01.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34009790 con 0x7fca440ff8b0 2023-08-25T04:42:01.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3400c810 con 0x7fca440ff8b0 2023-08-25T04:42:01.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca441a0ad0 con 0x7fca440ff8b0 2023-08-25T04:42:01.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34003c70 con 0x7fca440ff8b0 2023-08-25T04:42:01.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.838+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441a1020 con 0x7fca440ff8b0 2023-08-25T04:42:01.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.839+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4410b110 con 0x7fca440ff8b0 2023-08-25T04:42:01.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.841+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fca3400cc80 con 0x7fca440ff8b0 2023-08-25T04:42:01.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.842+0000 7fca417fa700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 0x7fca2c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:01.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.843+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fca34014070 con 0x7fca440ff8b0 2023-08-25T04:42:01.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.843+0000 7fca437fe700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 0x7fca2c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:01.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.843+0000 7fca437fe700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 0x7fca2c0a2490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca38011490 tx=0x7fca38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:01.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:01.847+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fca340950b0 con 0x7fca440ff8b0 2023-08-25T04:42:01.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:01 smithi162 ceph-mon[130941]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:01 smithi162 ceph-mon[130941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-25T04:42:02.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.060+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fca441a11b0 con 0x7fca440ff8b0 2023-08-25T04:42:02.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.061+0000 7fca417fa700 1 -- 172.21.15.145:0/500727097 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fca340986d0 con 0x7fca440ff8b0 2023-08-25T04:42:02.062 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:02.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.063+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 msgr2=0x7fca2c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:02.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.063+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 0x7fca2c0a2490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca38011490 tx=0x7fca38009260 comp rx=0 tx=0).stop 2023-08-25T04:42:02.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.063+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 msgr2=0x7fca44198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:02.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.063+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e9a0 tx=0x7fca3400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:42:02.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.063+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 shutdown_connections 2023-08-25T04:42:02.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fca2c09ffd0 0x7fca2c0a2490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:02.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44101450 0x7fca4419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:02.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fca440ff8b0 0x7fca44198990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:02.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 --2- 172.21.15.145:0/500727097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca44100ab0 0x7fca44198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:02.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 >> 172.21.15.145:0/500727097 conn(0x7fca440fae30 msgr2=0x7fca440fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:02.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 shutdown_connections 2023-08-25T04:42:02.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:02.064+0000 7fca4a902700 1 -- 172.21.15.145:0/500727097 wait complete. 2023-08-25T04:42:02.531 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1692938503,"num_in_osds":12,"osd_in_since":1692938506,"num_remapped_pgs":0} 2023-08-25T04:42:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T04:42:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T04:42:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: Detected new or changed devices on smithi183 2023-08-25T04:42:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 4292M 2023-08-25T04:42:02.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.831 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.832 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:02 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/500727097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T04:42:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T04:42:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: Detected new or changed devices on smithi183 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 4292M 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:02.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:02 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/500727097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:02.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T04:42:02.540+0000 7f11304e5700 -1 osd.11 0 waiting for initial osdmap 2023-08-25T04:42:02.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T04:42:02.554+0000 7f112aad7700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: osdmap e76: 12 total, 11 up, 12 in 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2023-08-25T04:42:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: Detected new or changed devices on smithi183 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 4292M 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:02.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:02.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:02.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:02 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/500727097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:03.533 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd stat -f json 2023-08-25T04:42:03.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:03 smithi145 ceph-mon[97681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:42:03.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:03 smithi145 ceph-mon[97681]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T04:42:03.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:03.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:03 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:03.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:03 smithi162 ceph-mon[130941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:42:03.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:03 smithi162 ceph-mon[130941]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T04:42:03.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:03.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:03 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:03.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:03 smithi183 ceph-mon[131414]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2023-08-25T04:42:03.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:03 smithi183 ceph-mon[131414]: osdmap e77: 12 total, 11 up, 12 in 2023-08-25T04:42:03.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:03.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:03 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:42:04.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: purged_snaps scrub starts 2023-08-25T04:42:04.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: purged_snaps scrub ok 2023-08-25T04:42:04.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] boot 2023-08-25T04:42:04.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T04:42:04.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:04 smithi145 ceph-mon[97681]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:04.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: purged_snaps scrub starts 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: purged_snaps scrub ok 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] boot 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:04 smithi162 ceph-mon[130941]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:04.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: purged_snaps scrub starts 2023-08-25T04:42:04.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: purged_snaps scrub ok 2023-08-25T04:42:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: osd.11 [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] boot 2023-08-25T04:42:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: osdmap e78: 12 total, 12 up, 12 in 2023-08-25T04:42:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:42:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:04 smithi183 ceph-mon[131414]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-25T04:42:05.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.617+0000 7f9656cd5700 1 -- 172.21.15.145:0/2761227803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 msgr2=0x7f9650108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.617+0000 7f9656cd5700 1 --2- 172.21.15.145:0/2761227803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650108e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f964800b0d0 tx=0x7f964800b3e0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 -- 172.21.15.145:0/2761227803 shutdown_connections 2023-08-25T04:42:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 --2- 172.21.15.145:0/2761227803 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650108e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 --2- 172.21.15.145:0/2761227803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96501009b0 0x7f9650100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 --2- 172.21.15.145:0/2761227803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9650101bb0 0x7f9650102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 -- 172.21.15.145:0/2761227803 >> 172.21.15.145:0/2761227803 conn(0x7f96500fbf10 msgr2=0x7f96500fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:05.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 -- 172.21.15.145:0/2761227803 shutdown_connections 2023-08-25T04:42:05.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.618+0000 7f9656cd5700 1 -- 172.21.15.145:0/2761227803 wait complete. 2023-08-25T04:42:05.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.619+0000 7f9656cd5700 1 Processor -- start 2023-08-25T04:42:05.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.619+0000 7f9656cd5700 1 -- start start 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96501009b0 0x7f96500746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9650101bb0 0x7f9650072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500739d0 con 0x7f9650102550 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650073b40 con 0x7f96501009b0 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9656cd5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501a07a0 con 0x7f9650101bb0 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9655272700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:05.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9655272700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57852/0 (socket says 172.21.15.145:57852) 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9654a71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96501009b0 0x7f96500746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f9655272700 1 -- 172.21.15.145:0/3066692966 learned_addr learned my addr 172.21.15.145:0/3066692966 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.620+0000 7f964ffff700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9650101bb0 0x7f9650072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 -- 172.21.15.145:0/3066692966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9650101bb0 msgr2=0x7f9650072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9650101bb0 0x7f9650072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 -- 172.21.15.145:0/3066692966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96501009b0 msgr2=0x7f96500746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:05.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96501009b0 0x7f96500746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 -- 172.21.15.145:0/3066692966 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009d00 con 0x7f9650102550 2023-08-25T04:42:05.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9655272700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9648008ec0 tx=0x7f9648008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:05.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800e040 con 0x7f9650102550 2023-08-25T04:42:05.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9648005070 con 0x7f9650102550 2023-08-25T04:42:05.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964801b6c0 con 0x7f9650102550 2023-08-25T04:42:05.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501a0a20 con 0x7f9650102550 2023-08-25T04:42:05.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.621+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501a0de0 con 0x7f9650102550 2023-08-25T04:42:05.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.622+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650107f30 con 0x7f9650102550 2023-08-25T04:42:05.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.624+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f96480051e0 con 0x7f9650102550 2023-08-25T04:42:05.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.626+0000 7f964dffb700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 0x7f96400a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:05.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.626+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96480cc400 con 0x7f9650102550 2023-08-25T04:42:05.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.626+0000 7f9654a71700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 0x7f96400a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:05.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.627+0000 7f9654a71700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 0x7f96400a2410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9650101a10 tx=0x7f9644009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:05.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.628+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9648096620 con 0x7f9650102550 2023-08-25T04:42:05.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:05 smithi145 ceph-mon[97681]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T04:42:05.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.828+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f965010c3d0 con 0x7f9650102550 2023-08-25T04:42:05.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.829+0000 7f964dffb700 1 -- 172.21.15.145:0/3066692966 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9648099c40 con 0x7f9650102550 2023-08-25T04:42:05.830 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:05.832 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 msgr2=0x7f96400a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:05.832 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 0x7f96400a2410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9650101a10 tx=0x7f9644009260 comp rx=0 tx=0).stop 2023-08-25T04:42:05.832 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 msgr2=0x7f9650073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9648008ec0 tx=0x7f9648008ef0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 shutdown_connections 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f964009ff50 0x7f96400a2410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650102550 0x7f9650073250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96501009b0 0x7f96500746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.831+0000 7f9656cd5700 1 --2- 172.21.15.145:0/3066692966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9650101bb0 0x7f9650072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:05.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.832+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 >> 172.21.15.145:0/3066692966 conn(0x7f96500fbf10 msgr2=0x7f9650105070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:05.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.832+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 shutdown_connections 2023-08-25T04:42:05.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:05.832+0000 7f9656cd5700 1 -- 172.21.15.145:0/3066692966 wait complete. 2023-08-25T04:42:05.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:05 smithi162 ceph-mon[130941]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T04:42:05.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:05 smithi183 ceph-mon[131414]: osdmap e79: 12 total, 12 up, 12 in 2023-08-25T04:42:06.244 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1692938523,"num_in_osds":12,"osd_in_since":1692938506,"num_remapped_pgs":0} 2023-08-25T04:42:06.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T04:42:06.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:06 smithi145 ceph-mon[97681]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:06 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3066692966' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:06.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:06 smithi162 ceph-mon[130941]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:06 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3066692966' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:06.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:06 smithi183 ceph-mon[131414]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:06.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:06 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3066692966' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-25T04:42:06.967 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:42:08.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 -- 172.21.15.145:0/3211130418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 msgr2=0x7f7330106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:08.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 --2- 172.21.15.145:0/3211130418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f7330106d20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f732c009990 tx=0x7f732c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 -- 172.21.15.145:0/3211130418 shutdown_connections 2023-08-25T04:42:08.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 --2- 172.21.15.145:0/3211130418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330101300 0x7f733010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 --2- 172.21.15.145:0/3211130418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f7330106d20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 --2- 172.21.15.145:0/3211130418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330100940 0x7f7330100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 -- 172.21.15.145:0/3211130418 >> 172.21.15.145:0/3211130418 conn(0x7f73300fc410 msgr2=0x7f73300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:08.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 -- 172.21.15.145:0/3211130418 shutdown_connections 2023-08-25T04:42:08.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.309+0000 7f7338490700 1 -- 172.21.15.145:0/3211130418 wait complete. 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.310+0000 7f7338490700 1 Processor -- start 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.310+0000 7f7338490700 1 -- start start 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.310+0000 7f7338490700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 0x7f7330072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.310+0000 7f7338490700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 0x7f7330073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7338490700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f73300736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7338490700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301a4e90 con 0x7f7330106940 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7338490700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301a5000 con 0x7f7330101300 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7338490700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301a51c0 con 0x7f7330100940 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7335a2b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 0x7f7330073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 0x7f7330072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7335a2b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 0x7f7330073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:33786/0 (socket says 172.21.15.145:33786) 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7336a2d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f73300736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7335a2b700 1 -- 172.21.15.145:0/3799074447 learned_addr learned my addr 172.21.15.145:0/3799074447 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:08.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 -- 172.21.15.145:0/3799074447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 msgr2=0x7f7330073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:08.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 0x7f7330073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 -- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 msgr2=0x7f73300736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:08.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f73300736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 -- 172.21.15.145:0/3799074447 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0095f0 con 0x7f7330100940 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f7336a2d700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f73300736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.311+0000 7f733622c700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 0x7f7330072c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.312+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01d070 con 0x7f7330100940 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.312+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73301a5440 con 0x7f7330100940 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.312+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c021410 con 0x7f7330100940 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.312+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c017650 con 0x7f7330100940 2023-08-25T04:42:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.312+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73301a58b0 con 0x7f7330100940 2023-08-25T04:42:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.313+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733004e620 con 0x7f7330100940 2023-08-25T04:42:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.315+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f732c0217f0 con 0x7f7330100940 2023-08-25T04:42:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.316+0000 7f73277fe700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 0x7f731c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.316+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f732c0cc7d0 con 0x7f7330100940 2023-08-25T04:42:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.317+0000 7f7335a2b700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 0x7f731c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.317+0000 7f7335a2b700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 0x7f731c0a2410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73300743a0 tx=0x7f7320009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.318+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f732c0969f0 con 0x7f7330100940 2023-08-25T04:42:08.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.515+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7330110920 con 0x7f7330100940 2023-08-25T04:42:08.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.517+0000 7f73277fe700 1 -- 172.21.15.145:0/3799074447 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18468 (secure 0 0 0) 0x7f732c09a010 con 0x7f7330100940 2023-08-25T04:42:08.519 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:08.519 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":79,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:04.548924+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:08.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.519+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 msgr2=0x7f731c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:08.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.519+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 0x7f731c0a2410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73300743a0 tx=0x7f7320009260 comp rx=0 tx=0).stop 2023-08-25T04:42:08.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.519+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 msgr2=0x7f7330072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:08.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.519+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 0x7f7330072c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c005740 comp rx=0 tx=0).stop 2023-08-25T04:42:08.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 shutdown_connections 2023-08-25T04:42:08.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f731c09ff50 0x7f731c0a2410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7330106940 0x7f73300736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7330100940 0x7f7330072c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 --2- 172.21.15.145:0/3799074447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7330101300 0x7f7330073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:08.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 >> 172.21.15.145:0/3799074447 conn(0x7f73300fc410 msgr2=0x7f73300fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:08.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 shutdown_connections 2023-08-25T04:42:08.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:08.520+0000 7f7338490700 1 -- 172.21.15.145:0/3799074447 wait complete. 2023-08-25T04:42:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:08 smithi145 ceph-mon[97681]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3799074447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:09.047 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T04:42:09.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-08-25T04:42:09.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:08 smithi162 ceph-mon[130941]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3799074447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:09.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:08 smithi183 ceph-mon[131414]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:09.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3799074447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:09.767 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:42:10.799 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:10 smithi145 ceph-mon[97681]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:10.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:10 smithi162 ceph-mon[130941]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:10.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:10 smithi183 ceph-mon[131414]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:11.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 -- 172.21.15.145:0/3292384716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260106940 msgr2=0x7ff260106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3292384716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260106940 0x7ff260106d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff250009990 tx=0x7ff250009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 -- 172.21.15.145:0/3292384716 shutdown_connections 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3292384716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 0x7ff26010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3292384716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260106940 0x7ff260106d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3292384716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260100940 0x7ff260100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.035+0000 7ff2669e3700 1 -- 172.21.15.145:0/3292384716 >> 172.21.15.145:0/3292384716 conn(0x7ff2600fc410 msgr2=0x7ff2600fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:11.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.036+0000 7ff2669e3700 1 -- 172.21.15.145:0/3292384716 shutdown_connections 2023-08-25T04:42:11.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.036+0000 7ff2669e3700 1 -- 172.21.15.145:0/3292384716 wait complete. 2023-08-25T04:42:11.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 Processor -- start 2023-08-25T04:42:11.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 -- start start 2023-08-25T04:42:11.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:11.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 0x7ff260073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 0x7ff2600736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a4ee0 con 0x7ff260100940 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a5050 con 0x7ff260106940 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff2669e3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a5210 con 0x7ff260101300 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff257fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:11.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff257fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57888/0 (socket says 172.21.15.145:57888) 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff257fff700 1 -- 172.21.15.145:0/3842069737 learned_addr learned my addr 172.21.15.145:0/3842069737 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.037+0000 7ff25ffff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 0x7ff260073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff264f80700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 0x7ff2600736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 -- 172.21.15.145:0/3842069737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 msgr2=0x7ff260073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 0x7ff260073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 -- 172.21.15.145:0/3842069737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 msgr2=0x7ff2600736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:11.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 0x7ff2600736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 -- 172.21.15.145:0/3842069737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2500095f0 con 0x7ff260100940 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff264f80700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 0x7ff2600736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff257fff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2500095c0 tx=0x7ff25000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25001d070 con 0x7ff260100940 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff250005050 con 0x7ff260100940 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250004b30 con 0x7ff260100940 2023-08-25T04:42:11.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.038+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2601a5490 con 0x7ff260100940 2023-08-25T04:42:11.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.039+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2601a58a0 con 0x7ff260100940 2023-08-25T04:42:11.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.040+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26004e620 con 0x7ff260100940 2023-08-25T04:42:11.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.041+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ff250004c90 con 0x7ff260100940 2023-08-25T04:42:11.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.043+0000 7ff25dffb700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 0x7ff24c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:11.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.043+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2500cbc60 con 0x7ff260100940 2023-08-25T04:42:11.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.043+0000 7ff25ffff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 0x7ff24c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:11.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.044+0000 7ff25ffff700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 0x7ff24c0a2490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff260074410 tx=0x7ff24800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:11.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.047+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff250095e80 con 0x7ff260100940 2023-08-25T04:42:11.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.244+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff260110b50 con 0x7ff260100940 2023-08-25T04:42:11.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.244+0000 7ff25dffb700 1 -- 172.21.15.145:0/3842069737 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2500994a0 con 0x7ff260100940 2023-08-25T04:42:11.245 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:42:11.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 msgr2=0x7ff24c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:11.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 0x7ff24c0a2490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff260074410 tx=0x7ff24800b410 comp rx=0 tx=0).stop 2023-08-25T04:42:11.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 msgr2=0x7ff260072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:11.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2500095c0 tx=0x7ff25000f690 comp rx=0 tx=0).stop 2023-08-25T04:42:11.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 shutdown_connections 2023-08-25T04:42:11.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff24c09ffd0 0x7ff24c0a2490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff260106940 0x7ff2600736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff260100940 0x7ff260072c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 --2- 172.21.15.145:0/3842069737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff260101300 0x7ff260073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:11.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.246+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 >> 172.21.15.145:0/3842069737 conn(0x7ff2600fc410 msgr2=0x7ff2600fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:11.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.247+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 shutdown_connections 2023-08-25T04:42:11.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:11.247+0000 7ff2669e3700 1 -- 172.21.15.145:0/3842069737 wait complete. 2023-08-25T04:42:11.639 INFO:tasks.cephadm:Adding mds.a on smithi145 2023-08-25T04:42:11.640 INFO:tasks.cephadm:Adding mds.d on smithi145 2023-08-25T04:42:11.640 INFO:tasks.cephadm:Adding mds.g on smithi145 2023-08-25T04:42:11.640 INFO:tasks.cephadm:Adding mds.j on smithi145 2023-08-25T04:42:11.640 INFO:tasks.cephadm:Adding mds.b on smithi162 2023-08-25T04:42:11.640 INFO:tasks.cephadm:Adding mds.e on smithi162 2023-08-25T04:42:11.641 INFO:tasks.cephadm:Adding mds.h on smithi162 2023-08-25T04:42:11.641 INFO:tasks.cephadm:Adding mds.k on smithi162 2023-08-25T04:42:11.641 INFO:tasks.cephadm:Adding mds.c on smithi183 2023-08-25T04:42:11.641 INFO:tasks.cephadm:Adding mds.f on smithi183 2023-08-25T04:42:11.641 INFO:tasks.cephadm:Adding mds.i on smithi183 2023-08-25T04:42:11.642 INFO:tasks.cephadm:Adding mds.l on smithi183 2023-08-25T04:42:11.642 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph orch apply mds all '12;smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2023-08-25T04:42:11.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:11 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3842069737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:11.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:11 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3842069737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:11.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:11 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3842069737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:12.211 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T04:42:12.787 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:12 smithi183 ceph-mon[131414]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:12.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:12 smithi145 ceph-mon[97681]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:12.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:12 smithi162 ceph-mon[130941]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:13.386 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 -- 172.21.15.183:0/1302992413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb0102550 msgr2=0x7effb0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:13.386 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 --2- 172.21.15.183:0/1302992413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb0102550 0x7effb0108e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effa800b0a0 tx=0x7effa800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.386 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 -- 172.21.15.183:0/1302992413 shutdown_connections 2023-08-25T04:42:13.386 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 --2- 172.21.15.183:0/1302992413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb0102550 0x7effb0108e00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 --2- 172.21.15.183:0/1302992413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb01009b0 0x7effb0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 --2- 172.21.15.183:0/1302992413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0101bb0 0x7effb0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 -- 172.21.15.183:0/1302992413 >> 172.21.15.183:0/1302992413 conn(0x7effb00fbf10 msgr2=0x7effb00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 -- 172.21.15.183:0/1302992413 shutdown_connections 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.383+0000 7effb7086700 1 -- 172.21.15.183:0/1302992413 wait complete. 2023-08-25T04:42:13.387 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 Processor -- start 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 -- start start 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb0101bb0 0x7effb0196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0102550 0x7effb019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb019e3e0 con 0x7effb0102550 2023-08-25T04:42:13.388 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb019e520 con 0x7effb0101bb0 2023-08-25T04:42:13.389 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb7086700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb019e6b0 con 0x7effb01009b0 2023-08-25T04:42:13.389 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb4e22700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:13.389 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb4e22700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52606/0 (socket says 172.21.15.183:52606) 2023-08-25T04:42:13.389 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb4e22700 1 -- 172.21.15.183:0/471758385 learned_addr learned my addr 172.21.15.183:0/471758385 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-08-25T04:42:13.389 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effaffff700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb0101bb0 0x7effb0196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.384+0000 7effb5623700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0102550 0x7effb019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 -- 172.21.15.183:0/471758385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb0101bb0 msgr2=0x7effb0196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb0101bb0 0x7effb0196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 -- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0102550 msgr2=0x7effb019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0102550 0x7effb019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 -- 172.21.15.183:0/471758385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa8009d00 con 0x7effb01009b0 2023-08-25T04:42:13.390 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb4e22700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effa400c920 tx=0x7effa400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:13.392 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4004220 con 0x7effb01009b0 2023-08-25T04:42:13.392 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa400ce70 con 0x7effb01009b0 2023-08-25T04:42:13.392 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb019e990 con 0x7effb01009b0 2023-08-25T04:42:13.392 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4003c00 con 0x7effb01009b0 2023-08-25T04:42:13.392 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.385+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb019eec0 con 0x7effb01009b0 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.386+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0107f50 con 0x7effb01009b0 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.387+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7effa4004380 con 0x7effb01009b0 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.387+0000 7effadffb700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 0x7effa00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.388+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7effa40ca9b0 con 0x7effb01009b0 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.388+0000 7effaffff700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 0x7effa00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:13.393 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.388+0000 7effaffff700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 0x7effa00a2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009a10 tx=0x7eff9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:13.394 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.392+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7effa4094be0 con 0x7effb01009b0 2023-08-25T04:42:13.591 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.590+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) v1 -- 0x7effb002cdc0 con 0x7effa009ff50 2023-08-25T04:42:13.600 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.599+0000 7effadffb700 1 -- 172.21.15.183:0/471758385 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7effb002cdc0 con 0x7effa009ff50 2023-08-25T04:42:13.601 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2023-08-25T04:42:13.602 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 msgr2=0x7effa00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 0x7effa00a2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009a10 tx=0x7eff9c009190 comp rx=0 tx=0).stop 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 msgr2=0x7effb0196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effa400c920 tx=0x7effa400cc30 comp rx=0 tx=0).stop 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 shutdown_connections 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7effa009ff50 0x7effa00a2410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effb0102550 0x7effb019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.603 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb01009b0 0x7effb0196850 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 --2- 172.21.15.183:0/471758385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effb0101bb0 0x7effb0196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:13.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 >> 172.21.15.183:0/471758385 conn(0x7effb00fbf10 msgr2=0x7effb0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:13.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 shutdown_connections 2023-08-25T04:42:13.604 INFO:teuthology.orchestra.run.smithi183.stderr:2023-08-25T04:42:13.601+0000 7effb7086700 1 -- 172.21.15.183:0/471758385 wait complete. 2023-08-25T04:42:13.954 DEBUG:teuthology.orchestra.run.smithi145:mds.a> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.a.service 2023-08-25T04:42:13.960 DEBUG:teuthology.orchestra.run.smithi145:mds.d> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.d.service 2023-08-25T04:42:14.006 DEBUG:teuthology.orchestra.run.smithi145:mds.g> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.g.service 2023-08-25T04:42:14.042 INFO:journalctl@ceph.mds.a.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:42:14.046 DEBUG:teuthology.orchestra.run.smithi145:mds.j> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.j.service 2023-08-25T04:42:14.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:42:14.082 DEBUG:teuthology.orchestra.run.smithi162:mds.b> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.b.service 2023-08-25T04:42:14.085 DEBUG:teuthology.orchestra.run.smithi162:mds.e> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.e.service 2023-08-25T04:42:14.116 INFO:journalctl@ceph.mds.g.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:42:14.122 INFO:journalctl@ceph.mds.b.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:42:14.123 DEBUG:teuthology.orchestra.run.smithi162:mds.h> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.h.service 2023-08-25T04:42:14.153 INFO:journalctl@ceph.mds.j.smithi145.stdout:-- Logs begin at Fri 2023-08-25 04:17:34 UTC. -- 2023-08-25T04:42:14.153 INFO:journalctl@ceph.mds.e.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:42:14.154 DEBUG:teuthology.orchestra.run.smithi162:mds.k> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.k.service 2023-08-25T04:42:14.184 INFO:journalctl@ceph.mds.h.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:42:14.185 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.c.service 2023-08-25T04:42:14.187 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.f.service 2023-08-25T04:42:14.210 INFO:journalctl@ceph.mds.k.smithi162.stdout:-- Logs begin at Fri 2023-08-25 04:12:54 UTC. -- 2023-08-25T04:42:14.218 INFO:journalctl@ceph.mds.c.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:42:14.219 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.i.service 2023-08-25T04:42:14.244 INFO:journalctl@ceph.mds.f.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:42:14.245 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.l.service 2023-08-25T04:42:14.270 INFO:journalctl@ceph.mds.i.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:42:14.271 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-08-25T04:42:14.272 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:42:14.299 INFO:journalctl@ceph.mds.l.smithi183.stdout:-- Logs begin at Fri 2023-08-25 04:12:18 UTC. -- 2023-08-25T04:42:14.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 -- 172.21.15.145:0/3675779729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 msgr2=0x7fee0c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 --2- 172.21.15.145:0/3675779729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c101f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009990 tx=0x7fedfc009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 -- 172.21.15.145:0/3675779729 shutdown_connections 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 --2- 172.21.15.145:0/3675779729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 0x7fee0c108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 --2- 172.21.15.145:0/3675779729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 0x7fee0c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 --2- 172.21.15.145:0/3675779729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c101f90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 -- 172.21.15.145:0/3675779729 >> 172.21.15.145:0/3675779729 conn(0x7fee0c0fbe90 msgr2=0x7fee0c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:14.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 -- 172.21.15.145:0/3675779729 shutdown_connections 2023-08-25T04:42:14.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.486+0000 7fee13dff700 1 -- 172.21.15.145:0/3675779729 wait complete. 2023-08-25T04:42:14.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.487+0000 7fee13dff700 1 Processor -- start 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.487+0000 7fee13dff700 1 -- start start 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 0x7fee0c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 0x7fee0c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0739f0 con 0x7fee0c1024d0 2023-08-25T04:42:14.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c073b60 con 0x7fee0c100940 2023-08-25T04:42:14.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee13dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1a06e0 con 0x7fee0c101b30 2023-08-25T04:42:14.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49872/0 (socket says 172.21.15.145:49872) 2023-08-25T04:42:14.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 -- 172.21.15.145:0/819808457 learned_addr learned my addr 172.21.15.145:0/819808457 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:14.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1239c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 0x7fee0c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee11b9b700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 0x7fee0c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 -- 172.21.15.145:0/819808457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 msgr2=0x7fee0c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 0x7fee0c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 -- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 msgr2=0x7fee0c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 0x7fee0c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.488+0000 7fee1139a700 1 -- 172.21.15.145:0/819808457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee04009c40 con 0x7fee0c101b30 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee1139a700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fedfc004740 tx=0x7fedfc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:14.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc01d070 con 0x7fee0c101b30 2023-08-25T04:42:14.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedfc004830 con 0x7fee0c101b30 2023-08-25T04:42:14.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0053c0 con 0x7fee0c101b30 2023-08-25T04:42:14.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc0095f0 con 0x7fee0c101b30 2023-08-25T04:42:14.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.489+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a0b40 con 0x7fee0c101b30 2023-08-25T04:42:14.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.490+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c107ee0 con 0x7fee0c101b30 2023-08-25T04:42:14.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.492+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fedfc005580 con 0x7fee0c101b30 2023-08-25T04:42:14.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.493+0000 7fee02ffd700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 0x7fedf80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.493+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fedfc0cbf20 con 0x7fee0c101b30 2023-08-25T04:42:14.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.493+0000 7fee11b9b700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 0x7fedf80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.494+0000 7fee11b9b700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 0x7fedf80a24e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee0c101990 tx=0x7fee08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:14.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.496+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fedfc096140 con 0x7fee0c101b30 2023-08-25T04:42:14.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.704+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee0c10c370 con 0x7fee0c101b30 2023-08-25T04:42:14.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.705+0000 7fee02ffd700 1 -- 172.21.15.145:0/819808457 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18468 (secure 0 0 0) 0x7fedfc099760 con 0x7fee0c101b30 2023-08-25T04:42:14.706 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:14.707 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":79,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:04.548924+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:14.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 msgr2=0x7fedf80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 0x7fedf80a24e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee0c101990 tx=0x7fee08009260 comp rx=0 tx=0).stop 2023-08-25T04:42:14.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 msgr2=0x7fee0c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fedfc004740 tx=0x7fedfc009580 comp rx=0 tx=0).stop 2023-08-25T04:42:14.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 shutdown_connections 2023-08-25T04:42:14.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fedf80a0020 0x7fedf80a24e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee0c1024d0 0x7fee0c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee0c100940 0x7fee0c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 --2- 172.21.15.145:0/819808457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee0c101b30 0x7fee0c072d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 >> 172.21.15.145:0/819808457 conn(0x7fee0c0fbe90 msgr2=0x7fee0c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:14.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 shutdown_connections 2023-08-25T04:42:14.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.708+0000 7fee13dff700 1 -- 172.21.15.145:0/819808457 wait complete. 2023-08-25T04:42:14.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T04:42:14.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:42:14.789 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:42:14.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: Saving service mds.all spec with placement smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2023-08-25T04:42:14.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:14.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:14.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:14.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: Deploying daemon mds.l on smithi183 2023-08-25T04:42:14.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:14 smithi183 ceph-mon[131414]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:42:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: Saving service mds.all spec with placement smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2023-08-25T04:42:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:14.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:14.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: Deploying daemon mds.l on smithi183 2023-08-25T04:42:14.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:14 smithi162 ceph-mon[130941]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:14.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 -- 172.21.15.145:0/1774036281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4101b20 msgr2=0x7f02e4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/1774036281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4101b20 0x7f02e4101f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009990 tx=0x7f02d4009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 -- 172.21.15.145:0/1774036281 shutdown_connections 2023-08-25T04:42:14.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/1774036281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e41024c0 0x7f02e4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/1774036281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e4100930 0x7f02e4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/1774036281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4101b20 0x7f02e4101f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 -- 172.21.15.145:0/1774036281 >> 172.21.15.145:0/1774036281 conn(0x7f02e40fbea0 msgr2=0x7f02e40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:14.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 -- 172.21.15.145:0/1774036281 shutdown_connections 2023-08-25T04:42:14.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.928+0000 7f02eb3ff700 1 -- 172.21.15.145:0/1774036281 wait complete. 2023-08-25T04:42:14.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 Processor -- start 2023-08-25T04:42:14.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 -- start start 2023-08-25T04:42:14.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e4101b20 0x7f02e4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e41024c0 0x7f02e4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40739f0 con 0x7f02e41024c0 2023-08-25T04:42:14.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4073b60 con 0x7f02e4101b20 2023-08-25T04:42:14.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.929+0000 7f02eb3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41a0680 con 0x7f02e4100930 2023-08-25T04:42:14.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e999c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e41024c0 0x7f02e4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49894/0 (socket says 172.21.15.145:49894) 2023-08-25T04:42:14.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 -- 172.21.15.145:0/765800552 learned_addr learned my addr 172.21.15.145:0/765800552 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:14.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e899a700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e4101b20 0x7f02e4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 -- 172.21.15.145:0/765800552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e4101b20 msgr2=0x7f02e4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e4101b20 0x7f02e4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 -- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e41024c0 msgr2=0x7f02e4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:14.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e41024c0 0x7f02e4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:14.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 -- 172.21.15.145:0/765800552 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40095f0 con 0x7f02e4100930 2023-08-25T04:42:14.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02e919b700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02e000e970 tx=0x7f02e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:14.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0009790 con 0x7f02e4100930 2023-08-25T04:42:14.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e000c7e0 con 0x7f02e4100930 2023-08-25T04:42:14.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e41a0960 con 0x7f02e4100930 2023-08-25T04:42:14.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.930+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0003c70 con 0x7f02e4100930 2023-08-25T04:42:14.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.931+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e41a0e70 con 0x7f02e4100930 2023-08-25T04:42:14.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.931+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e410c190 con 0x7f02e4100930 2023-08-25T04:42:14.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.934+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f02e000cc50 con 0x7f02e4100930 2023-08-25T04:42:14.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.935+0000 7f02da7fc700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 0x7f02d00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:14.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.935+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02e0014070 con 0x7f02e4100930 2023-08-25T04:42:14.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.935+0000 7f02e899a700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 0x7f02d00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:14.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.936+0000 7f02e899a700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 0x7f02d00a2410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b2e0 tx=0x7f02d4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:14.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:14.942+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f02e00952c0 con 0x7f02e4100930 2023-08-25T04:42:15.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:42:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: Saving service mds.all spec with placement smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2023-08-25T04:42:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: Deploying daemon mds.l on smithi183 2023-08-25T04:42:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:14 smithi145 ceph-mon[97681]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:15.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.141+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02e4107850 con 0x7f02e4100930 2023-08-25T04:42:15.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.142+0000 7f02da7fc700 1 -- 172.21.15.145:0/765800552 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02e0003dd0 con 0x7f02e4100930 2023-08-25T04:42:15.143 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:42:15.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 msgr2=0x7f02d00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 0x7f02d00a2410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b2e0 tx=0x7f02d4005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 msgr2=0x7f02e40746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02e000e970 tx=0x7f02e000ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:15.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 shutdown_connections 2023-08-25T04:42:15.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f02d009ff50 0x7f02d00a2410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02e41024c0 0x7f02e4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e4100930 0x7f02e40746d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 --2- 172.21.15.145:0/765800552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e4101b20 0x7f02e4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.144+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 >> 172.21.15.145:0/765800552 conn(0x7f02e40fbea0 msgr2=0x7f02e40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:15.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.145+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 shutdown_connections 2023-08-25T04:42:15.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.145+0000 7f02eb3ff700 1 -- 172.21.15.145:0/765800552 wait complete. 2023-08-25T04:42:15.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:15.172 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 systemd[1]: Starting Ceph mds.l for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:15.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1454999692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0100930 msgr2=0x7f7ac0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1454999692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0100930 0x7f7ac0100d50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00099c0 tx=0x7f7ab0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1454999692 shutdown_connections 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1454999692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1454999692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0100930 0x7f7ac0100d50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1454999692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0101b20 0x7f7ac0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.361+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1454999692 >> 172.21.15.145:0/1454999692 conn(0x7f7ac00fbe60 msgr2=0x7f7ac00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:15.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.362+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1454999692 shutdown_connections 2023-08-25T04:42:15.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.362+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1454999692 wait complete. 2023-08-25T04:42:15.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.363+0000 7f7ac49ff700 1 Processor -- start 2023-08-25T04:42:15.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.363+0000 7f7ac49ff700 1 -- start start 2023-08-25T04:42:15.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 0x7f7ac01965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:15.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:15.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:15.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019e170 con 0x7f7ac0101b20 2023-08-25T04:42:15.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019e2b0 con 0x7f7ac0100930 2023-08-25T04:42:15.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019e440 con 0x7f7ac01024c0 2023-08-25T04:42:15.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abdd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:15.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 0x7f7ac01965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:15.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abdd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60500/0 (socket says 172.21.15.145:60500) 2023-08-25T04:42:15.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abdd9b700 1 -- 172.21.15.145:0/1597932669 learned_addr learned my addr 172.21.15.145:0/1597932669 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:15.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abed9d700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:15.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 -- 172.21.15.145:0/1597932669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 msgr2=0x7f7ac019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 -- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 msgr2=0x7f7ac0196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 -- 172.21.15.145:0/1597932669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009520 con 0x7f7ac0100930 2023-08-25T04:42:15.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abdd9b700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:15.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abed9d700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:15.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.364+0000 7f7abe59c700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 0x7f7ac01965f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00098e0 tx=0x7f7ab000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:15.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.365+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab001d070 con 0x7f7ac0100930 2023-08-25T04:42:15.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.365+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00095f0 con 0x7f7ac0100930 2023-08-25T04:42:15.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.365+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab0005510 con 0x7f7ac0100930 2023-08-25T04:42:15.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.365+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0004b30 con 0x7f7ac0100930 2023-08-25T04:42:15.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.365+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac019e950 con 0x7f7ac0100930 2023-08-25T04:42:15.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.366+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac0107860 con 0x7f7ac0100930 2023-08-25T04:42:15.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.368+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f7ab0004c90 con 0x7f7ac0100930 2023-08-25T04:42:15.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.369+0000 7f7aaf7fe700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:15.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.369+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ab00cbbf0 con 0x7f7ac0100930 2023-08-25T04:42:15.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.369+0000 7f7abdd9b700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:15.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.370+0000 7f7abdd9b700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 0x7f7aa80a2460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009fd0 tx=0x7f7ab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:15.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.372+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7ab0095e10 con 0x7f7ac0100930 2023-08-25T04:42:15.601 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 podman[156515]: 2023-08-25T04:42:15.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.617+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ac010c3e0 con 0x7f7ac0100930 2023-08-25T04:42:15.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.618+0000 7f7aaf7fe700 1 -- 172.21.15.145:0/1597932669 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f7ab0099430 con 0x7f7ac0100930 2023-08-25T04:42:15.619 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:15.619 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-08-25T04:42:15.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.620+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 msgr2=0x7f7aa80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.620+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 0x7f7aa80a2460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009fd0 tx=0x7f7ab4009260 comp rx=0 tx=0).stop 2023-08-25T04:42:15.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.620+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 msgr2=0x7f7ac01965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.620+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 0x7f7ac01965f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00098e0 tx=0x7f7ab000f690 comp rx=0 tx=0).stop 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.620+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 shutdown_connections 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac01024c0 0x7f7ac019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac0100930 0x7f7ac01965f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 --2- 172.21.15.145:0/1597932669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ac0101b20 0x7f7ac0196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:15.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 >> 172.21.15.145:0/1597932669 conn(0x7f7ac00fbe60 msgr2=0x7f7ac00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:15.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 shutdown_connections 2023-08-25T04:42:15.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:15.621+0000 7f7ac49ff700 1 -- 172.21.15.145:0/1597932669 wait complete. 2023-08-25T04:42:15.623 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 1 2023-08-25T04:42:15.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:15 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/819808457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:15.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:15 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/765800552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:15.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:15 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1597932669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:15.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:15 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/819808457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:15.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:15 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/765800552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:15.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:15 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1597932669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:15.923 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 bash[156515]: 176afc5069b6cd368bd3c79030efedaceebfbacc9111382fd9174b1e1be1465e 2023-08-25T04:42:15.923 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 ceph-mds[156555]: -- 172.21.15.183:0/2084051701 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a099ccb1e0 con 0x55a099ca0800 2023-08-25T04:42:15.923 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 systemd[1]: Started Ceph mds.l for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:15.924 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 04:42:15 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-l[156531]: starting mds.l at 2023-08-25T04:42:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:15 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/819808457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:15 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/765800552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:15 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1597932669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:16.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6832/916411904,v1:172.21.15.183:6833/916411904] up:boot 2023-08-25T04:42:16.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: fsmap 1 up:standby 2023-08-25T04:42:16.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:42:16.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:16.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:16 smithi162 ceph-mon[130941]: Deploying daemon mds.k on smithi162 2023-08-25T04:42:16.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6832/916411904,v1:172.21.15.183:6833/916411904] up:boot 2023-08-25T04:42:16.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: fsmap 1 up:standby 2023-08-25T04:42:16.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:42:16.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:16.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:16.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:16.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:16.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:16.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:16 smithi183 ceph-mon[131414]: Deploying daemon mds.k on smithi162 2023-08-25T04:42:17.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6832/916411904,v1:172.21.15.183:6833/916411904] up:boot 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: fsmap 1 up:standby 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:17.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:17.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:17.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:17.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:16 smithi145 ceph-mon[97681]: Deploying daemon mds.k on smithi162 2023-08-25T04:42:17.412 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 systemd[1]: Starting Ceph mds.k for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:17.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:17.789 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 podman[155143]: 2023-08-25T04:42:17.790 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 bash[155143]: 7b429094ee57928b13152bbd8307001ec4db30ee95687565003572912eeba732 2023-08-25T04:42:17.790 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 ceph-mds[155183]: -- 172.21.15.162:0/1694716007 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8f6bff1e0 con 0x55a8f6bd4c00 2023-08-25T04:42:17.790 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-k[155159]: starting mds.k at 2023-08-25T04:42:17.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.850+0000 7f3fae5ff700 1 -- 172.21.15.145:0/2838037961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8073300 msgr2=0x7f3fa8073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.850+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/2838037961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8073300 0x7f3fa8073780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009990 tx=0x7f3f90009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 -- 172.21.15.145:0/2838037961 shutdown_connections 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/2838037961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa8108850 0x7f3fa810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/2838037961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa80748e0 0x7f3fa8072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/2838037961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8073300 0x7f3fa8073780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 -- 172.21.15.145:0/2838037961 >> 172.21.15.145:0/2838037961 conn(0x7f3fa80fbed0 msgr2=0x7f3fa80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:17.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 -- 172.21.15.145:0/2838037961 shutdown_connections 2023-08-25T04:42:17.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.851+0000 7f3fae5ff700 1 -- 172.21.15.145:0/2838037961 wait complete. 2023-08-25T04:42:17.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 Processor -- start 2023-08-25T04:42:17.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 -- start start 2023-08-25T04:42:17.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa8073300 0x7f3fa81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:17.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:17.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8108850 0x7f3fa819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:17.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81a0370 con 0x7f3fa80748e0 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81a04b0 con 0x7f3fa8108850 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fae5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81a0640 con 0x7f3fa8073300 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3f9ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3f9ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60510/0 (socket says 172.21.15.145:60510) 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3f9ffff700 1 -- 172.21.15.145:0/747486308 learned_addr learned my addr 172.21.15.145:0/747486308 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:17.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3fa7fff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa8073300 0x7f3fa81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:17.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.852+0000 7f3facb9c700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8108850 0x7f3fa819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:17.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 -- 172.21.15.145:0/747486308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa8073300 msgr2=0x7f3fa81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:17.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa8073300 0x7f3fa81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 -- 172.21.15.145:0/747486308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8108850 msgr2=0x7f3fa819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:17.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8108850 0x7f3fa819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:17.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 -- 172.21.15.145:0/747486308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900095f0 con 0x7f3fa80748e0 2023-08-25T04:42:17.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3f9ffff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005910 tx=0x7f3f9000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9001d070 con 0x7f3fa80748e0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f90022470 con 0x7f3fa80748e0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90017650 con 0x7f3fa80748e0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa81a08c0 con 0x7f3fa80748e0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.853+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa81a0ce0 con 0x7f3fa80748e0 2023-08-25T04:42:17.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.854+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa8106f50 con 0x7f3fa80748e0 2023-08-25T04:42:17.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.856+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3f9000fad0 con 0x7f3fa80748e0 2023-08-25T04:42:17.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.858+0000 7f3fa5ffb700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 0x7f3f940a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:17.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.858+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f900cc0b0 con 0x7f3fa80748e0 2023-08-25T04:42:17.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.859+0000 7f3fa7fff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 0x7f3f940a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:17.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.859+0000 7f3fa7fff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 0x7f3f940a24e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f98007c60 tx=0x7f3f980073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:17.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:17.863+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3f900962d0 con 0x7f3fa80748e0 2023-08-25T04:42:18.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.133+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fa810e3a0 con 0x7f3fa80748e0 2023-08-25T04:42:18.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.133+0000 7f3fa5ffb700 1 -- 172.21.15.145:0/747486308 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f3f900998f0 con 0x7f3fa80748e0 2023-08-25T04:42:18.135 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:18.135 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 msgr2=0x7f3f940a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 0x7f3f940a24e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f98007c60 tx=0x7f3f980073d0 comp rx=0 tx=0).stop 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 msgr2=0x7f3fa8198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005910 tx=0x7f3f9000f690 comp rx=0 tx=0).stop 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 shutdown_connections 2023-08-25T04:42:18.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f940a0020 0x7f3f940a24e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa8108850 0x7f3fa819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fa8073300 0x7f3fa81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 --2- 172.21.15.145:0/747486308 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3fa80748e0 0x7f3fa8198d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 >> 172.21.15.145:0/747486308 conn(0x7f3fa80fbed0 msgr2=0x7f3fa80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 shutdown_connections 2023-08-25T04:42:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:18.136+0000 7f3fae5ff700 1 -- 172.21.15.145:0/747486308 wait complete. 2023-08-25T04:42:18.140 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 3 2023-08-25T04:42:18.161 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 04:42:17 smithi162 systemd[1]: Started Ceph mds.k for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:19.033 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:19.033 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6832/4170975809,v1:172.21.15.162:6833/4170975809] up:boot 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: fsmap 2 up:standby 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:19.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:19.035 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:19.035 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: Deploying daemon mds.j on smithi145 2023-08-25T04:42:19.035 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:18 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/747486308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6832/4170975809,v1:172.21.15.162:6833/4170975809] up:boot 2023-08-25T04:42:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: fsmap 2 up:standby 2023-08-25T04:42:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:42:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: Deploying daemon mds.j on smithi145 2023-08-25T04:42:19.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:18 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/747486308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6832/4170975809,v1:172.21.15.162:6833/4170975809] up:boot 2023-08-25T04:42:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: fsmap 2 up:standby 2023-08-25T04:42:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:42:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: Deploying daemon mds.j on smithi145 2023-08-25T04:42:19.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:18 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/747486308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:20.078 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:19 smithi145 systemd[1]: Starting Ceph mds.j for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:20.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 -- 172.21.15.145:0/3122650015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8102b20 msgr2=0x7f9da8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 --2- 172.21.15.145:0/3122650015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8102b20 0x7f9da8102f40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9d980099c0 tx=0x7f9d98009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 -- 172.21.15.145:0/3122650015 shutdown_connections 2023-08-25T04:42:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 --2- 172.21.15.145:0/3122650015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da81046b0 0x7f9da810af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 --2- 172.21.15.145:0/3122650015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8102b20 0x7f9da8102f40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 --2- 172.21.15.145:0/3122650015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da8103d10 0x7f9da8104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 -- 172.21.15.145:0/3122650015 >> 172.21.15.145:0/3122650015 conn(0x7f9da80fe070 msgr2=0x7f9da81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:20.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 -- 172.21.15.145:0/3122650015 shutdown_connections 2023-08-25T04:42:20.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.371+0000 7f9daddff700 1 -- 172.21.15.145:0/3122650015 wait complete. 2023-08-25T04:42:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.372+0000 7f9daddff700 1 Processor -- start 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.372+0000 7f9daddff700 1 -- start start 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da8102b20 0x7f9da81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 0x7f9da819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da81a0360 con 0x7f9da8103d10 2023-08-25T04:42:20.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da81a04a0 con 0x7f9da81046b0 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9daddff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da81a0630 con 0x7f9da8102b20 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60530/0 (socket says 172.21.15.145:60530) 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 -- 172.21.15.145:0/901050586 learned_addr learned my addr 172.21.15.145:0/901050586 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da77fe700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da8102b20 0x7f9da81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da7fff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 0x7f9da819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 -- 172.21.15.145:0/901050586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da8102b20 msgr2=0x7f9da81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da8102b20 0x7f9da81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 -- 172.21.15.145:0/901050586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 msgr2=0x7f9da819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 0x7f9da819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da6ffd700 1 -- 172.21.15.145:0/901050586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980095f0 con 0x7f9da8103d10 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.373+0000 7f9da7fff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 0x7f9da819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9da6ffd700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00e9a0 tx=0x7f9d9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c009790 con 0x7f9da8103d10 2023-08-25T04:42:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9c00cbd0 con 0x7f9da8103d10 2023-08-25T04:42:20.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da81a0910 con 0x7f9da8103d10 2023-08-25T04:42:20.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c018680 con 0x7f9da8103d10 2023-08-25T04:42:20.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.374+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da81a0e70 con 0x7f9da8103d10 2023-08-25T04:42:20.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.375+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da810e380 con 0x7f9da8103d10 2023-08-25T04:42:20.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.377+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9d9c00cd40 con 0x7f9da8103d10 2023-08-25T04:42:20.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.379+0000 7f9da4ff9700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 0x7f9d900a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:20.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.379+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9d9c014070 con 0x7f9da8103d10 2023-08-25T04:42:20.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.379+0000 7f9da77fe700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 0x7f9d900a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.380+0000 7f9da77fe700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 0x7f9d900a24e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009990 tx=0x7f9d98005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:20.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.383+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9d9c051e00 con 0x7f9da8103d10 2023-08-25T04:42:20.514 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:20 smithi145 podman[130949]: 2023-08-25T04:42:20.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.641+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9da8109a20 con 0x7f9da8103d10 2023-08-25T04:42:20.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.641+0000 7f9da4ff9700 1 -- 172.21.15.145:0/901050586 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f9d9c095170 con 0x7f9da8103d10 2023-08-25T04:42:20.643 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:20.643 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T04:42:20.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 msgr2=0x7f9d900a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:20.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 0x7f9d900a24e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009990 tx=0x7f9d98005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 msgr2=0x7f9da8198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:20.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00e9a0 tx=0x7f9d9c00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 shutdown_connections 2023-08-25T04:42:20.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d900a0020 0x7f9d900a24e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9da81046b0 0x7f9da819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da8102b20 0x7f9da81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 --2- 172.21.15.145:0/901050586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9da8103d10 0x7f9da8198d20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 >> 172.21.15.145:0/901050586 conn(0x7f9da80fe070 msgr2=0x7f9da81002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 shutdown_connections 2023-08-25T04:42:20.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:20.644+0000 7f9daddff700 1 -- 172.21.15.145:0/901050586 wait complete. 2023-08-25T04:42:20.648 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 3 2023-08-25T04:42:20.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:20 smithi145 ceph-mon[97681]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:20.828 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:20 smithi145 bash[130949]: 9d8be3e6fcec1bfd1c38a66679bad771a28605583f974038e58908c3aba28208 2023-08-25T04:42:20.828 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:20 smithi145 ceph-mds[131019]: -- 172.21.15.145:0/349847677 <== mon.0 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5638975f11e0 con 0x5638975c6800 2023-08-25T04:42:20.828 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-j[130995]: starting mds.j at 2023-08-25T04:42:20.828 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 04:42:20 smithi145 systemd[1]: Started Ceph mds.j for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:20.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:20 smithi162 ceph-mon[130941]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:20.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:20 smithi183 ceph-mon[131414]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:21.802 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/901050586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:21.802 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:21.803 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:21 smithi145 ceph-mon[97681]: Deploying daemon mds.d on smithi145 2023-08-25T04:42:21.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/901050586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:21.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:21.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:21 smithi162 ceph-mon[130941]: Deploying daemon mds.d on smithi145 2023-08-25T04:42:21.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/901050586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:21.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:21.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:21 smithi183 ceph-mon[131414]: Deploying daemon mds.d on smithi145 2023-08-25T04:42:22.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:22.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:22 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6834/4172912052,v1:172.21.15.145:6835/4172912052] up:boot 2023-08-25T04:42:22.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:22 smithi145 ceph-mon[97681]: fsmap 3 up:standby 2023-08-25T04:42:22.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:22 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:42:22.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:22 smithi145 ceph-mon[97681]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:22.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:22 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6834/4172912052,v1:172.21.15.145:6835/4172912052] up:boot 2023-08-25T04:42:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:22 smithi162 ceph-mon[130941]: fsmap 3 up:standby 2023-08-25T04:42:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:22 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:42:22.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:22 smithi162 ceph-mon[130941]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:22.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:22 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6834/4172912052,v1:172.21.15.145:6835/4172912052] up:boot 2023-08-25T04:42:22.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:22 smithi183 ceph-mon[131414]: fsmap 3 up:standby 2023-08-25T04:42:22.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:22 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:42:22.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:22 smithi183 ceph-mon[131414]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 -- 172.21.15.145:0/2775485990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260101980 msgr2=0x7f4260101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 --2- 172.21.15.145:0/2775485990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260101980 0x7f4260101de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4254009990 tx=0x7f4254009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 -- 172.21.15.145:0/2775485990 shutdown_connections 2023-08-25T04:42:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 --2- 172.21.15.145:0/2775485990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260102320 0x7f4260108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 --2- 172.21.15.145:0/2775485990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260100790 0x7f4260100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 --2- 172.21.15.145:0/2775485990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260101980 0x7f4260101de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 -- 172.21.15.145:0/2775485990 >> 172.21.15.145:0/2775485990 conn(0x7f42600fbce0 msgr2=0x7f42600fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 -- 172.21.15.145:0/2775485990 shutdown_connections 2023-08-25T04:42:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.041+0000 7f425f59e700 1 -- 172.21.15.145:0/2775485990 wait complete. 2023-08-25T04:42:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.042+0000 7f425f59e700 1 Processor -- start 2023-08-25T04:42:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.042+0000 7f425f59e700 1 -- start start 2023-08-25T04:42:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 0x7f4260198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 0x7f426019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42601a00e0 con 0x7f4260100790 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42601a0220 con 0x7f4260101980 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425f59e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42601a03b0 con 0x7f4260102320 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37992/0 (socket says 172.21.15.145:37992) 2023-08-25T04:42:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 -- 172.21.15.145:0/3028150001 learned_addr learned my addr 172.21.15.145:0/3028150001 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425dd9b700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 0x7f4260198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425ed9d700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 0x7f426019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 -- 172.21.15.145:0/3028150001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 msgr2=0x7f426019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 0x7f426019fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 -- 172.21.15.145:0/3028150001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 msgr2=0x7f4260198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 0x7f4260198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425e59c700 1 -- 172.21.15.145:0/3028150001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009c40 con 0x7f4260100790 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425ed9d700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 0x7f426019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.043+0000 7f425dd9b700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 0x7f4260198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f425e59c700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f425000e970 tx=0x7f425000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250009790 con 0x7f4260100790 2023-08-25T04:42:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f425000eec0 con 0x7f4260100790 2023-08-25T04:42:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42540095f0 con 0x7f4260100790 2023-08-25T04:42:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250003cf0 con 0x7f4260100790 2023-08-25T04:42:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.044+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42601a0a60 con 0x7f4260100790 2023-08-25T04:42:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.045+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426010c000 con 0x7f4260100790 2023-08-25T04:42:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.047+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f425000ccd0 con 0x7f4260100790 2023-08-25T04:42:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.049+0000 7f424f7fe700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 0x7f42480a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.049+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4250014070 con 0x7f4260100790 2023-08-25T04:42:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.049+0000 7f425dd9b700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 0x7f42480a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.049+0000 7f425dd9b700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 0x7f42480a2580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f425400b2e0 tx=0x7f4254005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:23.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.053+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4250095180 con 0x7f4260100790 2023-08-25T04:42:23.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.319+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42601a0ce0 con 0x7f4260100790 2023-08-25T04:42:23.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.320+0000 7f424f7fe700 1 -- 172.21.15.145:0/3028150001 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f42500987a0 con 0x7f4260100790 2023-08-25T04:42:23.322 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:23.322 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T04:42:23.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 msgr2=0x7f42480a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:23.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 0x7f42480a2580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f425400b2e0 tx=0x7f4254005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 msgr2=0x7f4260198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f425000e970 tx=0x7f425000ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 shutdown_connections 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f42480a00c0 0x7f42480a2580 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4260102320 0x7f426019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4260100790 0x7f4260198560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 --2- 172.21.15.145:0/3028150001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4260101980 0x7f4260198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:23.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.322+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 >> 172.21.15.145:0/3028150001 conn(0x7f42600fbce0 msgr2=0x7f4260107110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:23.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.323+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 shutdown_connections 2023-08-25T04:42:23.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:23.323+0000 7f425f59e700 1 -- 172.21.15.145:0/3028150001 wait complete. 2023-08-25T04:42:23.327 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 4 2023-08-25T04:42:23.328 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:22 smithi145 systemd[1]: Starting Ceph mds.d for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:23.642 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:23 smithi145 podman[131365]: 2023-08-25T04:42:23.642 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:23 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3028150001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:23.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:23 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3028150001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:23.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:23 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3028150001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:24.077 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:23 smithi145 bash[131365]: 77c21fb1862e387f4682d0d1b3d63bca6f8e864b66dd40b9684eeb95be9edbfd 2023-08-25T04:42:24.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:23 smithi145 ceph-mds[131407]: -- 172.21.15.145:0/1359354914 <== mon.0 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555daa331e0 con 0x5555daa09000 2023-08-25T04:42:24.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:23 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-d[131383]: starting mds.d at 2023-08-25T04:42:24.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 04:42:23 smithi145 systemd[1]: Started Ceph mds.d for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:25.048 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:25.049 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.049 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.049 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.049 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:25.049 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:25.050 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:25.050 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: Deploying daemon mds.e on smithi162 2023-08-25T04:42:25.050 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6836/4182229769,v1:172.21.15.145:6837/4182229769] up:boot 2023-08-25T04:42:25.050 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: fsmap 4 up:standby 2023-08-25T04:42:25.050 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:24 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:42:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:25.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: Deploying daemon mds.e on smithi162 2023-08-25T04:42:25.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6836/4182229769,v1:172.21.15.145:6837/4182229769] up:boot 2023-08-25T04:42:25.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: fsmap 4 up:standby 2023-08-25T04:42:25.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:24 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:42:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: Deploying daemon mds.e on smithi162 2023-08-25T04:42:25.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6836/4182229769,v1:172.21.15.145:6837/4182229769] up:boot 2023-08-25T04:42:25.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: fsmap 4 up:standby 2023-08-25T04:42:25.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:24 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:42:25.345 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:25.412 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 systemd[1]: Starting Ceph mds.e for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:25.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 -- 172.21.15.145:0/219681956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 msgr2=0x7f88b0100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 --2- 172.21.15.145:0/219681956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0100d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88a00099c0 tx=0x7f88a0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 -- 172.21.15.145:0/219681956 shutdown_connections 2023-08-25T04:42:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 --2- 172.21.15.145:0/219681956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 0x7f88b0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 --2- 172.21.15.145:0/219681956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0100d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 --2- 172.21.15.145:0/219681956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 -- 172.21.15.145:0/219681956 >> 172.21.15.145:0/219681956 conn(0x7f88b00fbe50 msgr2=0x7f88b00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:25.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 -- 172.21.15.145:0/219681956 shutdown_connections 2023-08-25T04:42:25.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.553+0000 7f88b57ff700 1 -- 172.21.15.145:0/219681956 wait complete. 2023-08-25T04:42:25.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.555+0000 7f88b57ff700 1 Processor -- start 2023-08-25T04:42:25.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.555+0000 7f88b57ff700 1 -- start start 2023-08-25T04:42:25.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.555+0000 7f88b57ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:25.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.555+0000 7f88b57ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:25.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88b57ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 0x7f88b019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:25.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88b57ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019e1e0 con 0x7f88b0100920 2023-08-25T04:42:25.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88b57ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019e320 con 0x7f88b01024b0 2023-08-25T04:42:25.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88b57ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019e4b0 con 0x7f88b0101b10 2023-08-25T04:42:25.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:25.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88ae7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:25.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38026/0 (socket says 172.21.15.145:38026) 2023-08-25T04:42:25.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 -- 172.21.15.145:0/689226130 learned_addr learned my addr 172.21.15.145:0/689226130 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:25.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88af7fe700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 0x7f88b019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:25.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 -- 172.21.15.145:0/689226130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 msgr2=0x7f88b0196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:25.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 -- 172.21.15.145:0/689226130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 msgr2=0x7f88b019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:25.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 0x7f88b019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 -- 172.21.15.145:0/689226130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00095f0 con 0x7f88b0100920 2023-08-25T04:42:25.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88ae7fc700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:25.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.556+0000 7f88aeffd700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:25.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.557+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d070 con 0x7f88b0100920 2023-08-25T04:42:25.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.557+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a0005030 con 0x7f88b0100920 2023-08-25T04:42:25.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.557+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b019e730 con 0x7f88b0100920 2023-08-25T04:42:25.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.557+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004b30 con 0x7f88b0100920 2023-08-25T04:42:25.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.557+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b019eb50 con 0x7f88b0100920 2023-08-25T04:42:25.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.558+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b010c200 con 0x7f88b0100920 2023-08-25T04:42:25.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.560+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f88a0004cd0 con 0x7f88b0100920 2023-08-25T04:42:25.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.561+0000 7f8897fff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 0x7f88980a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:25.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.561+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88a00cbcb0 con 0x7f88b0100920 2023-08-25T04:42:25.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.561+0000 7f88ae7fc700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 0x7f88980a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:25.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.562+0000 7f88ae7fc700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 0x7f88980a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88a4011490 tx=0x7f88a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:25.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.565+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f88a0095ed0 con 0x7f88b0100920 2023-08-25T04:42:25.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.815+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88b01077d0 con 0x7f88b0100920 2023-08-25T04:42:25.817 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.816+0000 7f8897fff700 1 -- 172.21.15.145:0/689226130 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3707 (secure 0 0 0) 0x7f88a00994f0 con 0x7f88b0100920 2023-08-25T04:42:25.818 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:25.818 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-25T04:42:25.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 msgr2=0x7f88980a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:25.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 0x7f88980a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88a4011490 tx=0x7f88a4009260 comp rx=0 tx=0).stop 2023-08-25T04:42:25.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 msgr2=0x7f88b0196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:25.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a000f690 comp rx=0 tx=0).stop 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 shutdown_connections 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f889809ffa0 0x7f88980a2460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b01024b0 0x7f88b019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b0100920 0x7f88b0196660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 --2- 172.21.15.145:0/689226130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b0101b10 0x7f88b0196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 >> 172.21.15.145:0/689226130 conn(0x7f88b00fbe50 msgr2=0x7f88b00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:25.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.818+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 shutdown_connections 2023-08-25T04:42:25.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:25.819+0000 7f88b57ff700 1 -- 172.21.15.145:0/689226130 wait complete. 2023-08-25T04:42:25.822 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 5 2023-08-25T04:42:25.875 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 podman[155483]: 2023-08-25T04:42:25.875 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 bash[155483]: 6d9400e38ff3bdd3f37ccaec1fdea61db13e24a2105a5d45a84fcdf0a64a80a9 2023-08-25T04:42:25.876 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 ceph-mds[155523]: -- 172.21.15.162:0/3349791484 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56278195f1e0 con 0x562781934800 2023-08-25T04:42:25.876 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-e[155499]: starting mds.e at 2023-08-25T04:42:26.161 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 04:42:25 smithi162 systemd[1]: Started Ceph mds.e for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:26.901 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:26.901 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/689226130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:26.901 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:26.901 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:26.902 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:26.902 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:26.902 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:26.902 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:26.902 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:26 smithi183 ceph-mon[131414]: Deploying daemon mds.f on smithi183 2023-08-25T04:42:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/689226130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:27.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:27.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:27.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:27.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:26 smithi145 ceph-mon[97681]: Deploying daemon mds.f on smithi183 2023-08-25T04:42:27.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/689226130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:27.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:27.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:26 smithi162 ceph-mon[130941]: Deploying daemon mds.f on smithi183 2023-08-25T04:42:27.434 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:27 smithi183 systemd[1]: Starting Ceph mds.f for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:27.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:28.020 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:27 smithi183 podman[156866]: 2023-08-25T04:42:28.021 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:27 smithi183 bash[156866]: b85f8454c710453ad3c2e51cb1352ec25c66db646f07011a1df2a7a7395d21b8 2023-08-25T04:42:28.021 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:27 smithi183 ceph-mds[156906]: -- 172.21.15.183:0/4273683386 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fa2d7b1e0 con 0x560fa2d50c00 2023-08-25T04:42:28.021 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: starting mds.f at 2023-08-25T04:42:28.021 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:27 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6834/2094921807,v1:172.21.15.162:6835/2094921807] up:boot 2023-08-25T04:42:28.021 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:27 smithi183 ceph-mon[131414]: fsmap 5 up:standby 2023-08-25T04:42:28.022 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:27 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:42:28.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 -- 172.21.15.145:0/2523767202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834101b30 msgr2=0x7fc834101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:28.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 --2- 172.21.15.145:0/2523767202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834101b30 0x7fc834101f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8280099c0 tx=0x7fc828009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 -- 172.21.15.145:0/2523767202 shutdown_connections 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 --2- 172.21.15.145:0/2523767202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 0x7fc834108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 --2- 172.21.15.145:0/2523767202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834100940 0x7fc834100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 --2- 172.21.15.145:0/2523767202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834101b30 0x7fc834101f90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.047+0000 7fc8399ff700 1 -- 172.21.15.145:0/2523767202 >> 172.21.15.145:0/2523767202 conn(0x7fc8340fbe90 msgr2=0x7fc8340fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.048+0000 7fc8399ff700 1 -- 172.21.15.145:0/2523767202 shutdown_connections 2023-08-25T04:42:28.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.048+0000 7fc8399ff700 1 -- 172.21.15.145:0/2523767202 wait complete. 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.049+0000 7fc8399ff700 1 Processor -- start 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.049+0000 7fc8399ff700 1 -- start start 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834101b30 0x7fc834196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 0x7fc83419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:28.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419e140 con 0x7fc834100940 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419e280 con 0x7fc834101b30 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8399ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419e410 con 0x7fc8341024d0 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38036/0 (socket says 172.21.15.145:38036) 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 -- 172.21.15.145:0/1075479063 learned_addr learned my addr 172.21.15.145:0/1075479063 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8327fc700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834101b30 0x7fc834196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc8337fe700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 0x7fc83419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:28.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 -- 172.21.15.145:0/1075479063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 msgr2=0x7fc83419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:28.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 0x7fc83419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 -- 172.21.15.145:0/1075479063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834101b30 msgr2=0x7fc834196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:28.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834101b30 0x7fc834196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.050+0000 7fc832ffd700 1 -- 172.21.15.145:0/1075479063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280095f0 con 0x7fc834100940 2023-08-25T04:42:28.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc832ffd700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e970 tx=0x7fc82400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824009790 con 0x7fc834100940 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc82400c7e0 con 0x7fc834100940 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83419e6f0 con 0x7fc834100940 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824003c70 con 0x7fc834100940 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.051+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83419ec00 con 0x7fc834100940 2023-08-25T04:42:28.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.052+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc834107e90 con 0x7fc834100940 2023-08-25T04:42:28.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.054+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc82400cc50 con 0x7fc834100940 2023-08-25T04:42:28.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.056+0000 7fc8389fd700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 0x7fc81c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:28.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.056+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc824014070 con 0x7fc834100940 2023-08-25T04:42:28.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.056+0000 7fc8327fc700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 0x7fc81c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:28.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.057+0000 7fc8327fc700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 0x7fc81c0a2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b580 tx=0x7fc828005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:28.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.060+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc824095150 con 0x7fc834100940 2023-08-25T04:42:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:27 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6834/2094921807,v1:172.21.15.162:6835/2094921807] up:boot 2023-08-25T04:42:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:27 smithi145 ceph-mon[97681]: fsmap 5 up:standby 2023-08-25T04:42:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:27 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:42:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:27 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6834/2094921807,v1:172.21.15.162:6835/2094921807] up:boot 2023-08-25T04:42:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:27 smithi162 ceph-mon[130941]: fsmap 5 up:standby 2023-08-25T04:42:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:27 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:42:28.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.308+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc83410c1a0 con 0x7fc834100940 2023-08-25T04:42:28.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.309+0000 7fc8389fd700 1 -- 172.21.15.145:0/1075479063 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4501 (secure 0 0 0) 0x7fc824003dd0 con 0x7fc834100940 2023-08-25T04:42:28.311 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:28.311 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-08-25T04:42:28.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 msgr2=0x7fc81c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 0x7fc81c0a2460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b580 tx=0x7fc828005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 msgr2=0x7fc8341965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e970 tx=0x7fc82400ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 shutdown_connections 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc81c09ffa0 0x7fc81c0a2460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8341024d0 0x7fc83419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc834100940 0x7fc8341965c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 --2- 172.21.15.145:0/1075479063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc834101b30 0x7fc834196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:28.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 >> 172.21.15.145:0/1075479063 conn(0x7fc8340fbe90 msgr2=0x7fc834104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:28.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 shutdown_connections 2023-08-25T04:42:28.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:28.312+0000 7fc8399ff700 1 -- 172.21.15.145:0/1075479063 wait complete. 2023-08-25T04:42:28.316 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 6 2023-08-25T04:42:28.422 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 04:42:28 smithi183 systemd[1]: Started Ceph mds.f for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: Deploying daemon mds.c on smithi183 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1075479063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6834/557521219,v1:172.21.15.183:6835/557521219] up:boot 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: fsmap 6 up:standby 2023-08-25T04:42:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:29 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:42:29.335 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:29.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:29.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: Deploying daemon mds.c on smithi183 2023-08-25T04:42:29.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1075479063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:29.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6834/557521219,v1:172.21.15.183:6835/557521219] up:boot 2023-08-25T04:42:29.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: fsmap 6 up:standby 2023-08-25T04:42:29.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:29 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:42:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: Deploying daemon mds.c on smithi183 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1075479063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6834/557521219,v1:172.21.15.183:6835/557521219] up:boot 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: fsmap 6 up:standby 2023-08-25T04:42:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:29 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:42:29.586 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:29 smithi183 systemd[1]: Starting Ceph mds.c for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:30.140 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:29 smithi183 podman[157206]: 2023-08-25T04:42:30.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:30.422 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:30 smithi183 bash[157206]: aa26c26f98dbad623ba0b03732d051e9074fe7c30f9d940f68a317eb24546fb4 2023-08-25T04:42:30.423 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mds[157246]: -- 172.21.15.183:0/603933759 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5880dd1e0 con 0x55b5880b3000 2023-08-25T04:42:30.423 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: starting mds.c at 2023-08-25T04:42:30.423 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 04:42:30 smithi183 systemd[1]: Started Ceph mds.c for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:30.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.538+0000 7f812a3ff700 1 -- 172.21.15.145:0/3710255296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81241046f0 msgr2=0x7f812410afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:30.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.538+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3710255296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81241046f0 0x7f812410afa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b0d0 tx=0x7f811c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 -- 172.21.15.145:0/3710255296 shutdown_connections 2023-08-25T04:42:30.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3710255296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81241046f0 0x7f812410afa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3710255296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3710255296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8124103d50 0x7f81241041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 -- 172.21.15.145:0/3710255296 >> 172.21.15.145:0/3710255296 conn(0x7f81240fe0d0 msgr2=0x7f8124100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:30.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 -- 172.21.15.145:0/3710255296 shutdown_connections 2023-08-25T04:42:30.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.539+0000 7f812a3ff700 1 -- 172.21.15.145:0/3710255296 wait complete. 2023-08-25T04:42:30.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 Processor -- start 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 -- start start 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8124103d50 0x7f8124198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241a04b0 con 0x7f8124102b60 2023-08-25T04:42:30.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241a05f0 con 0x7f8124103d50 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.540+0000 7f812a3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241a0780 con 0x7f81241046f0 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f8123fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56394/0 (socket says 172.21.15.145:56394) 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 -- 172.21.15.145:0/3921898811 learned_addr learned my addr 172.21.15.145:0/3921898811 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f8123fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38048/0 (socket says 172.21.15.145:38048) 2023-08-25T04:42:30.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f81237fe700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8124103d50 0x7f8124198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:30.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 -- 172.21.15.145:0/3921898811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8124103d50 msgr2=0x7f8124198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:30.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8124103d50 0x7f8124198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 -- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 msgr2=0x7f8124198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:30.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 -- 172.21.15.145:0/3921898811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009d00 con 0x7f81241046f0 2023-08-25T04:42:30.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812899c700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f811c008eb0 tx=0x7f811c008ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:30.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00e040 con 0x7f81241046f0 2023-08-25T04:42:30.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811c00bdb0 con 0x7f81241046f0 2023-08-25T04:42:30.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c01d470 con 0x7f81241046f0 2023-08-25T04:42:30.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.541+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81241a0a00 con 0x7f81241046f0 2023-08-25T04:42:30.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.542+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81241a0e80 con 0x7f81241046f0 2023-08-25T04:42:30.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.543+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812410a000 con 0x7f81241046f0 2023-08-25T04:42:30.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.544+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f811c005020 con 0x7f81241046f0 2023-08-25T04:42:30.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.545+0000 7f81217fa700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 0x7f810c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:30.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.545+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f811c0cbd50 con 0x7f81241046f0 2023-08-25T04:42:30.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.546+0000 7f8123fff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 0x7f810c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:30.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.546+0000 7f8123fff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 0x7f810c0a24e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8124103bb0 tx=0x7f8114009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:30.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.549+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f811c095f70 con 0x7f81241046f0 2023-08-25T04:42:30.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.793+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f812410e3d0 con 0x7f81241046f0 2023-08-25T04:42:30.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.794+0000 7f81217fa700 1 -- 172.21.15.145:0/3921898811 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6086 (secure 0 0 0) 0x7f811c099590 con 0x7f81241046f0 2023-08-25T04:42:30.796 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-08-25T04:42:30.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 msgr2=0x7f810c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 0x7f810c0a24e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8124103bb0 tx=0x7f8114009260 comp rx=0 tx=0).stop 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 msgr2=0x7f812419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f811c008eb0 tx=0x7f811c008ee0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 shutdown_connections 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f810c0a0020 0x7f810c0a24e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81241046f0 0x7f812419fee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8124102b60 0x7f8124198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 --2- 172.21.15.145:0/3921898811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8124103d50 0x7f8124198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:30.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 >> 172.21.15.145:0/3921898811 conn(0x7f81240fe0d0 msgr2=0x7f8124107140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:30.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 shutdown_connections 2023-08-25T04:42:30.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:30.797+0000 7f812a3ff700 1 -- 172.21.15.145:0/3921898811 wait complete. 2023-08-25T04:42:30.800 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2023-08-25T04:42:30.815 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:30.815 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.815 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.815 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.815 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:30.816 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:30.816 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:30 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:30.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:30.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:30.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:30 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:30.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:30 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:31.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:31 smithi162 ceph-mon[130941]: Deploying daemon mds.h on smithi162 2023-08-25T04:42:31.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:31 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6836/4241662550,v1:172.21.15.183:6837/4241662550] up:boot 2023-08-25T04:42:31.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:31 smithi162 ceph-mon[130941]: fsmap 7 up:standby 2023-08-25T04:42:31.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:31 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:42:31.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:31 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3921898811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:31.782 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:31 smithi162 systemd[1]: Starting Ceph mds.h for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:31.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:31 smithi145 ceph-mon[97681]: Deploying daemon mds.h on smithi162 2023-08-25T04:42:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:31 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6836/4241662550,v1:172.21.15.183:6837/4241662550] up:boot 2023-08-25T04:42:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:31 smithi145 ceph-mon[97681]: fsmap 7 up:standby 2023-08-25T04:42:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:31 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:42:31.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:31 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3921898811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:31.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:31 smithi183 ceph-mon[131414]: Deploying daemon mds.h on smithi162 2023-08-25T04:42:31.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:31 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6836/4241662550,v1:172.21.15.183:6837/4241662550] up:boot 2023-08-25T04:42:31.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:31 smithi183 ceph-mon[131414]: fsmap 7 up:standby 2023-08-25T04:42:31.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:31 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:42:31.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:31 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3921898811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:32.356 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:32 smithi162 podman[155826]: 2023-08-25T04:42:32.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:32.663 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-mds[155866]: -- 172.21.15.162:0/860662669 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f124331e0 con 0x559f12409000 2023-08-25T04:42:32.663 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:32 smithi162 bash[155826]: 921b5f9fb2afc7f57beed7a633244a188d50b23fb4db0bf4ca6a801a90474e57 2023-08-25T04:42:32.663 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: starting mds.h at 2023-08-25T04:42:32.663 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 04:42:32 smithi162 systemd[1]: Started Ceph mds.h for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:32.817 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:32.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:32.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:32 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:32.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:32.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:32.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:32.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:32.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:32 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:33.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 -- 172.21.15.145:0/2368883983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c0100920 msgr2=0x7fe1c0100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:33.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/2368883983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c0100920 0x7fe1c0100d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009990 tx=0x7fe1b0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 -- 172.21.15.145:0/2368883983 shutdown_connections 2023-08-25T04:42:33.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/2368883983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c01024b0 0x7fe1c0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/2368883983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c0100920 0x7fe1c0100d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/2368883983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 -- 172.21.15.145:0/2368883983 >> 172.21.15.145:0/2368883983 conn(0x7fe1c00fbe50 msgr2=0x7fe1c00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:33.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 -- 172.21.15.145:0/2368883983 shutdown_connections 2023-08-25T04:42:33.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.024+0000 7fe1c55ff700 1 -- 172.21.15.145:0/2368883983 wait complete. 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.025+0000 7fe1c55ff700 1 Processor -- start 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.025+0000 7fe1c55ff700 1 -- start start 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.025+0000 7fe1c55ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1c55ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1c55ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c01024b0 0x7fe1c019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:33.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1c55ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a03c0 con 0x7fe1c0100920 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1c55ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a0500 con 0x7fe1c0101b10 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1c55ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a0690 con 0x7fe1c01024b0 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50452/0 (socket says 172.21.15.145:50452) 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 -- 172.21.15.145:0/1993146926 learned_addr learned my addr 172.21.15.145:0/1993146926 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1be7fc700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:33.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 -- 172.21.15.145:0/1993146926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c01024b0 msgr2=0x7fe1c019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c01024b0 0x7fe1c019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 -- 172.21.15.145:0/1993146926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 msgr2=0x7fe1c0198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 -- 172.21.15.145:0/1993146926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00095f0 con 0x7fe1c0100920 2023-08-25T04:42:33.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1be7fc700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:33.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.026+0000 7fe1beffd700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005720 tx=0x7fe1b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:33.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.027+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b001d070 con 0x7fe1c0100920 2023-08-25T04:42:33.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.027+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c01a0910 con 0x7fe1c0100920 2023-08-25T04:42:33.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.027+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1b0021410 con 0x7fe1c0100920 2023-08-25T04:42:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.027+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0017650 con 0x7fe1c0100920 2023-08-25T04:42:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.027+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c01a0d90 con 0x7fe1c0100920 2023-08-25T04:42:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.028+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c010c180 con 0x7fe1c0100920 2023-08-25T04:42:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.030+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fe1b0021580 con 0x7fe1c0100920 2023-08-25T04:42:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.032+0000 7fe1a7fff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 0x7fe1a80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.032+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1b00cc1a0 con 0x7fe1c0100920 2023-08-25T04:42:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.032+0000 7fe1be7fc700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 0x7fe1a80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.033+0000 7fe1be7fc700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 0x7fe1a80a2460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009cb0 tx=0x7fe1b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:33.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.037+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe1b0096440 con 0x7fe1c0100920 2023-08-25T04:42:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.277+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c01a0fb0 con 0x7fe1c0100920 2023-08-25T04:42:33.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.277+0000 7fe1a7fff700 1 -- 172.21.15.145:0/1993146926 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7fe1b0099a60 con 0x7fe1c0100920 2023-08-25T04:42:33.279 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:33.280 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-25T04:42:33.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 msgr2=0x7fe1a80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 0x7fe1a80a2460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009cb0 tx=0x7fe1b4009260 comp rx=0 tx=0).stop 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 msgr2=0x7fe1c0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005720 tx=0x7fe1b0009580 comp rx=0 tx=0).stop 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 shutdown_connections 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe1a809ffa0 0x7fe1a80a2460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c01024b0 0x7fe1c019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1c0100920 0x7fe1c0198840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 --2- 172.21.15.145:0/1993146926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1c0101b10 0x7fe1c0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:33.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 >> 172.21.15.145:0/1993146926 conn(0x7fe1c00fbe50 msgr2=0x7fe1c00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:33.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 shutdown_connections 2023-08-25T04:42:33.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:33.280+0000 7fe1c55ff700 1 -- 172.21.15.145:0/1993146926 wait complete. 2023-08-25T04:42:33.283 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 9 2023-08-25T04:42:33.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:33 smithi145 ceph-mon[97681]: Deploying daemon mds.i on smithi183 2023-08-25T04:42:33.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:33 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6836/1048214793,v1:172.21.15.162:6837/1048214793] up:boot 2023-08-25T04:42:33.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:33 smithi145 ceph-mon[97681]: fsmap 8 up:standby 2023-08-25T04:42:33.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:42:33.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:33 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1993146926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:33.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:33 smithi183 ceph-mon[131414]: Deploying daemon mds.i on smithi183 2023-08-25T04:42:33.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:33 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6836/1048214793,v1:172.21.15.162:6837/1048214793] up:boot 2023-08-25T04:42:33.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:33 smithi183 ceph-mon[131414]: fsmap 8 up:standby 2023-08-25T04:42:33.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:42:33.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:33 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1993146926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:33.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:33 smithi162 ceph-mon[130941]: Deploying daemon mds.i on smithi183 2023-08-25T04:42:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:33 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6836/1048214793,v1:172.21.15.162:6837/1048214793] up:boot 2023-08-25T04:42:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:33 smithi162 ceph-mon[130941]: fsmap 8 up:standby 2023-08-25T04:42:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:42:33.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:33 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1993146926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:34.173 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:33 smithi183 systemd[1]: Starting Ceph mds.i for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:34.522 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:34 smithi183 podman[157549]: 2023-08-25T04:42:34.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:34 smithi145 ceph-mon[97681]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:34.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:34 smithi162 ceph-mon[130941]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:34.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:34 smithi183 ceph-mon[131414]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:34.923 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:34 smithi183 ceph-mds[157590]: -- 172.21.15.183:0/3502050727 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2b3f371e0 con 0x55b2b3f0cc00 2023-08-25T04:42:34.923 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: starting mds.i at 2023-08-25T04:42:34.923 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:34 smithi183 bash[157549]: 4119f03ccf6763502601553a9a5531af3208901bc5d1bb4d93acb578bade3e59 2023-08-25T04:42:34.923 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 04:42:34 smithi183 systemd[1]: Started Ceph mds.i for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:35.302 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:35.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.514+0000 7fde5b1ff700 1 -- 172.21.15.145:0/3853735196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 msgr2=0x7fde5410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:35.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/3853735196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5410aca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00b0a0 tx=0x7fde4c00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 -- 172.21.15.145:0/3853735196 shutdown_connections 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/3853735196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5410aca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/3853735196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54074900 0x7fde54072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/3853735196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54073320 0x7fde540737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 -- 172.21.15.145:0/3853735196 >> 172.21.15.145:0/3853735196 conn(0x7fde540fbed0 msgr2=0x7fde540fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 -- 172.21.15.145:0/3853735196 shutdown_connections 2023-08-25T04:42:35.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.515+0000 7fde5b1ff700 1 -- 172.21.15.145:0/3853735196 wait complete. 2023-08-25T04:42:35.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 Processor -- start 2023-08-25T04:42:35.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 -- start start 2023-08-25T04:42:35.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54073320 0x7fde541988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:35.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54074900 0x7fde54198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a0430 con 0x7fde54108830 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a0570 con 0x7fde54073320 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.516+0000 7fde5b1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541a0700 con 0x7fde54074900 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50474/0 (socket says 172.21.15.145:50474) 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 -- 172.21.15.145:0/2873836426 learned_addr learned my addr 172.21.15.145:0/2873836426 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:35.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde53fff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54074900 0x7fde54198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde58f9b700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54073320 0x7fde541988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 -- 172.21.15.145:0/2873836426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54074900 msgr2=0x7fde54198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54074900 0x7fde54198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 -- 172.21.15.145:0/2873836426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54073320 msgr2=0x7fde541988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54073320 0x7fde541988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 -- 172.21.15.145:0/2873836426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c009d00 con 0x7fde54108830 2023-08-25T04:42:35.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde5979c700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009fd0 tx=0x7fde4c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:35.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.517+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00e040 con 0x7fde54108830 2023-08-25T04:42:35.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.518+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4c003e10 con 0x7fde54108830 2023-08-25T04:42:35.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.518+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde541a0980 con 0x7fde54108830 2023-08-25T04:42:35.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.518+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c01b6a0 con 0x7fde54108830 2023-08-25T04:42:35.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.518+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde541a0e00 con 0x7fde54108830 2023-08-25T04:42:35.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.519+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde54106f30 con 0x7fde54108830 2023-08-25T04:42:35.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.521+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fde4c01d470 con 0x7fde54108830 2023-08-25T04:42:35.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.522+0000 7fde51ffb700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 0x7fde440a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:35.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.522+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde4c0cbee0 con 0x7fde54108830 2023-08-25T04:42:35.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.522+0000 7fde58f9b700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 0x7fde440a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:35.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.523+0000 7fde58f9b700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 0x7fde440a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde48009c60 tx=0x7fde48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:35.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.527+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fde4c096100 con 0x7fde54108830 2023-08-25T04:42:35.645 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:boot 2023-08-25T04:42:35.645 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: fsmap 9 up:standby 2023-08-25T04:42:35.646 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:42:35.646 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.646 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.646 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.647 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:35.647 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:35.647 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:35.647 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:35 smithi145 ceph-mon[97681]: Deploying daemon mds.g on smithi145 2023-08-25T04:42:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.799+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde5402ccc0 con 0x7fde54108830 2023-08-25T04:42:35.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.799+0000 7fde51ffb700 1 -- 172.21.15.145:0/2873836426 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7fde4c099720 con 0x7fde54108830 2023-08-25T04:42:35.800 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:35.801 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T04:42:35.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 msgr2=0x7fde440a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 0x7fde440a24e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde48009c60 tx=0x7fde48009260 comp rx=0 tx=0).stop 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 msgr2=0x7fde5419fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009fd0 tx=0x7fde4c00bce0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 shutdown_connections 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fde440a0020 0x7fde440a24e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde54108830 0x7fde5419fe60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde54073320 0x7fde541988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 --2- 172.21.15.145:0/2873836426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde54074900 0x7fde54198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 >> 172.21.15.145:0/2873836426 conn(0x7fde540fbed0 msgr2=0x7fde5410aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 shutdown_connections 2023-08-25T04:42:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:35.802+0000 7fde5b1ff700 1 -- 172.21.15.145:0/2873836426 wait complete. 2023-08-25T04:42:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 10 2023-08-25T04:42:35.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:boot 2023-08-25T04:42:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: fsmap 9 up:standby 2023-08-25T04:42:35.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:42:35.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:35.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:35.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:35.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:35 smithi162 ceph-mon[130941]: Deploying daemon mds.g on smithi145 2023-08-25T04:42:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:boot 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: fsmap 9 up:standby 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:35.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:35.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:35.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:35 smithi183 ceph-mon[131414]: Deploying daemon mds.g on smithi145 2023-08-25T04:42:36.825 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:36 smithi145 ceph-mon[97681]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:36.825 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:36 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2873836426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:36.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:36 smithi162 ceph-mon[130941]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:36.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:36 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2873836426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:36.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:36 smithi183 ceph-mon[131414]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:36.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:36 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2873836426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:37.309 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:36 smithi145 systemd[1]: Starting Ceph mds.g for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:37.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:37 smithi145 podman[131931]: 2023-08-25T04:42:37.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:37 smithi145 bash[131931]: b3464942652e8a55862473cacb12ac05c8e7c8b6771d1fad7e2ed3cba877e453 2023-08-25T04:42:37.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:37 smithi145 ceph-mds[131971]: -- 172.21.15.145:0/3735084222 <== mon.0 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557a7e65d1e0 con 0x557a7e632c00 2023-08-25T04:42:37.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-g[131947]: starting mds.g at 2023-08-25T04:42:37.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:37.887 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 04:42:37 smithi145 systemd[1]: Started Ceph mds.g for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:38.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 -- 172.21.15.145:0/3287854281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 msgr2=0x7f8634072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:38.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 --2- 172.21.15.145:0/3287854281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634072db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86240098f0 tx=0x7f8624009c00 comp rx=0 tx=0).stop 2023-08-25T04:42:38.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 -- 172.21.15.145:0/3287854281 shutdown_connections 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 --2- 172.21.15.145:0/3287854281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 --2- 172.21.15.145:0/3287854281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634072db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 --2- 172.21.15.145:0/3287854281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 0x7f8634073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 -- 172.21.15.145:0/3287854281 >> 172.21.15.145:0/3287854281 conn(0x7f86340fbe60 msgr2=0x7f86340fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.030+0000 7f86399ff700 1 -- 172.21.15.145:0/3287854281 shutdown_connections 2023-08-25T04:42:38.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.031+0000 7f86399ff700 1 -- 172.21.15.145:0/3287854281 wait complete. 2023-08-25T04:42:38.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 Processor -- start 2023-08-25T04:42:38.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 -- start start 2023-08-25T04:42:38.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 0x7f8634198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:38.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:38.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341a03c0 con 0x7f86341087f0 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341a0500 con 0x7f8634073380 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.033+0000 7f86399ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341a0690 con 0x7f8634074960 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86337fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41208/0 (socket says 172.21.15.145:41208) 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 -- 172.21.15.145:0/2273129367 learned_addr learned my addr 172.21.15.145:0/2273129367 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:38.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86337fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50516/0 (socket says 172.21.15.145:50516) 2023-08-25T04:42:38.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f8632ffd700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 0x7f8634198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:38.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 -- 172.21.15.145:0/2273129367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 msgr2=0x7f8634198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:38.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 0x7f8634198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 -- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 msgr2=0x7f863419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:38.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 -- 172.21.15.145:0/2273129367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624009520 con 0x7f8634074960 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86337fe700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.034+0000 7f86327fc700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f862800e970 tx=0x7f862800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.035+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628009790 con 0x7f8634074960 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.035+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86341a0970 con 0x7f8634074960 2023-08-25T04:42:38.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.035+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862800c7e0 con 0x7f8634074960 2023-08-25T04:42:38.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.035+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628003c70 con 0x7f8634074960 2023-08-25T04:42:38.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.035+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86341a0e80 con 0x7f8634074960 2023-08-25T04:42:38.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.036+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8634106ea0 con 0x7f8634074960 2023-08-25T04:42:38.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.038+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f862800cc50 con 0x7f8634074960 2023-08-25T04:42:38.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.039+0000 7f86389fd700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 0x7f861c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:38.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.039+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8628014070 con 0x7f8634074960 2023-08-25T04:42:38.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.039+0000 7f8632ffd700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 0x7f861c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:38.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.040+0000 7f8632ffd700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 0x7f861c0a2460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8624005e90 tx=0x7f8624019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:38.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.043+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f86280ce030 con 0x7f8634074960 2023-08-25T04:42:38.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.321+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863410e370 con 0x7f8634074960 2023-08-25T04:42:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.323+0000 7f86389fd700 1 -- 172.21.15.145:0/2273129367 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8471 (secure 0 0 0) 0x7f8628095160 con 0x7f8634074960 2023-08-25T04:42:38.323 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:38.324 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T04:42:38.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.325+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 msgr2=0x7f861c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:38.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.325+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 0x7f861c0a2460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8624005e90 tx=0x7f8624019040 comp rx=0 tx=0).stop 2023-08-25T04:42:38.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.325+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 msgr2=0x7f8634198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.325+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f862800e970 tx=0x7f862800ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 shutdown_connections 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f861c09ffa0 0x7f861c0a2460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86341087f0 0x7f863419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8634073380 0x7f8634198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 --2- 172.21.15.145:0/2273129367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634074960 0x7f8634198d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:38.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 >> 172.21.15.145:0/2273129367 conn(0x7f86340fbe60 msgr2=0x7f86340fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:38.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 shutdown_connections 2023-08-25T04:42:38.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:38.326+0000 7f86399ff700 1 -- 172.21.15.145:0/2273129367 wait complete. 2023-08-25T04:42:38.329 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 11 2023-08-25T04:42:38.695 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6838/3652134728,v1:172.21.15.145:6839/3652134728] up:boot 2023-08-25T04:42:38.695 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: fsmap 10 up:standby 2023-08-25T04:42:38.695 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:42:38.695 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.695 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.696 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:38.696 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.697 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:38.697 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:38.697 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:38.697 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: Deploying daemon mds.a on smithi145 2023-08-25T04:42:38.697 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:38 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2273129367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:38.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6838/3652134728,v1:172.21.15.145:6839/3652134728] up:boot 2023-08-25T04:42:38.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: fsmap 10 up:standby 2023-08-25T04:42:38.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:42:38.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:38.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:38.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:38.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:38.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: Deploying daemon mds.a on smithi145 2023-08-25T04:42:38.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:38 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2273129367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:38.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6838/3652134728,v1:172.21.15.145:6839/3652134728] up:boot 2023-08-25T04:42:38.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: fsmap 10 up:standby 2023-08-25T04:42:38.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:42:38.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:38.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:38.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:38.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:38.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:38.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: Deploying daemon mds.a on smithi145 2023-08-25T04:42:38.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:38 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2273129367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:40.077 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:39 smithi145 systemd[1]: Starting Ceph mds.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:40.347 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:40.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 -- 172.21.15.145:0/2794933088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 msgr2=0x7f296c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:40.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 --2- 172.21.15.145:0/2794933088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c101f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f295c0099c0 tx=0x7f295c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 -- 172.21.15.145:0/2794933088 shutdown_connections 2023-08-25T04:42:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 --2- 172.21.15.145:0/2794933088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 --2- 172.21.15.145:0/2794933088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 --2- 172.21.15.145:0/2794933088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c101f80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.562+0000 7f29737ff700 1 -- 172.21.15.145:0/2794933088 >> 172.21.15.145:0/2794933088 conn(0x7f296c0fbe60 msgr2=0x7f296c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:40.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.563+0000 7f29737ff700 1 -- 172.21.15.145:0/2794933088 shutdown_connections 2023-08-25T04:42:40.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.563+0000 7f29737ff700 1 -- 172.21.15.145:0/2794933088 wait complete. 2023-08-25T04:42:40.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.564+0000 7f29737ff700 1 Processor -- start 2023-08-25T04:42:40.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.564+0000 7f29737ff700 1 -- start start 2023-08-25T04:42:40.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:40.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:40.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:40.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0739f0 con 0x7f296c100930 2023-08-25T04:42:40.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c073b60 con 0x7f296c1024c0 2023-08-25T04:42:40.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f29737ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1a06d0 con 0x7f296c101b20 2023-08-25T04:42:40.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f297159b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:40.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f297159b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50540/0 (socket says 172.21.15.145:50540) 2023-08-25T04:42:40.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f297159b700 1 -- 172.21.15.145:0/916882284 learned_addr learned my addr 172.21.15.145:0/916882284 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f2971d9c700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.565+0000 7f2970d9a700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 -- 172.21.15.145:0/916882284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 msgr2=0x7f296c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 -- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 msgr2=0x7f296c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 -- 172.21.15.145:0/916882284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0095f0 con 0x7f296c101b20 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f2970d9a700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 0x7f296c072d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f295c0098e0 tx=0x7f295c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c01d070 con 0x7f296c101b20 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f295c0054e0 con 0x7f296c101b20 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c004b70 con 0x7f296c101b20 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c1a0950 con 0x7f296c101b20 2023-08-25T04:42:40.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.566+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1a0cc0 con 0x7f296c101b20 2023-08-25T04:42:40.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.568+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c107ed0 con 0x7f296c101b20 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.570+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f295c00fc30 con 0x7f296c101b20 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.572+0000 7f29627fc700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 0x7f29580a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.572+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f295c0cb960 con 0x7f296c101b20 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.572+0000 7f297159b700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 0x7f29580a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.573+0000 7f297159b700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 0x7f29580a24e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2968005950 tx=0x7f2968009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:40.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.575+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f295c095b80 con 0x7f296c101b20 2023-08-25T04:42:40.627 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:40 smithi145 ceph-mon[97681]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:40.627 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:40 smithi145 podman[132318]: 2023-08-25T04:42:40.628 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:40 smithi145 bash[132318]: d41923750717dae1044a4d5ce210b9d4ec9b7e97f17f1a9180bc15dad473f9b3 2023-08-25T04:42:40.628 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:40 smithi145 ceph-mds[132381]: -- 172.21.15.145:0/3181321036 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bce566b1e0 con 0x55bce5640800 2023-08-25T04:42:40.628 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: starting mds.a at 2023-08-25T04:42:40.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.840+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f296c02cdc0 con 0x7f296c101b20 2023-08-25T04:42:40.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.841+0000 7f29627fc700 1 -- 172.21.15.145:0/916882284 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8471 (secure 0 0 0) 0x7f295c0991a0 con 0x7f296c101b20 2023-08-25T04:42:40.842 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:40.842 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-25T04:42:40.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 msgr2=0x7f29580a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:40.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 0x7f29580a24e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2968005950 tx=0x7f2968009260 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 msgr2=0x7f296c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 0x7f296c072d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f295c0098e0 tx=0x7f295c00f740 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 shutdown_connections 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f29580a0020 0x7f29580a24e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 --2- 172.21.15.145:0/916882284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:40.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 >> 172.21.15.145:0/916882284 conn(0x7f296c0fbe60 msgr2=0x7f296c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:40.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 shutdown_connections 2023-08-25T04:42:40.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:40.844+0000 7f29737ff700 1 -- 172.21.15.145:0/916882284 wait complete. 2023-08-25T04:42:40.847 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 11 2023-08-25T04:42:40.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:40 smithi162 ceph-mon[130941]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:40.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:40 smithi183 ceph-mon[131414]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:41.077 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:40 smithi145 systemd[1]: Started Ceph mds.a for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:41.781 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:41.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:41.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:41.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:41.782 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: Deploying daemon mds.b on smithi162 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/916882284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:boot 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: fsmap 11 up:standby 2023-08-25T04:42:41.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:41 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:42:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: Deploying daemon mds.b on smithi162 2023-08-25T04:42:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/916882284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:boot 2023-08-25T04:42:42.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: fsmap 11 up:standby 2023-08-25T04:42:42.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:41 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:42:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-25T04:42:42.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-25T04:42:42.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:42.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: Deploying daemon mds.b on smithi162 2023-08-25T04:42:42.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/916882284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:42.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:boot 2023-08-25T04:42:42.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: fsmap 11 up:standby 2023-08-25T04:42:42.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:41 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:42:42.338 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 systemd[1]: Starting Ceph mds.b for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T04:42:42.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:42.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:42 smithi162 ceph-mon[130941]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:42.912 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 podman[156165]: 2023-08-25T04:42:42.913 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 bash[156165]: 70d60d983197d35a93c4af31f0817290e1d8956075c79ad67b2b9d5eaf10f382 2023-08-25T04:42:42.913 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 ceph-mds[156205]: -- 172.21.15.162:0/3788670952 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587e0c7d1e0 con 0x5587e0c53000 2023-08-25T04:42:42.913 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: starting mds.b at 2023-08-25T04:42:43.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:42 smithi145 ceph-mon[97681]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:43.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 -- 172.21.15.145:0/949207153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 msgr2=0x7fdcfc101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:43.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 --2- 172.21.15.145:0/949207153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc101f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009920 tx=0x7fdcec009c30 comp rx=0 tx=0).stop 2023-08-25T04:42:43.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 -- 172.21.15.145:0/949207153 shutdown_connections 2023-08-25T04:42:43.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 --2- 172.21.15.145:0/949207153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 0x7fdcfc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 --2- 172.21.15.145:0/949207153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 --2- 172.21.15.145:0/949207153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc101f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 -- 172.21.15.145:0/949207153 >> 172.21.15.145:0/949207153 conn(0x7fdcfc0fbe60 msgr2=0x7fdcfc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:43.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 -- 172.21.15.145:0/949207153 shutdown_connections 2023-08-25T04:42:43.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.080+0000 7fdd037ff700 1 -- 172.21.15.145:0/949207153 wait complete. 2023-08-25T04:42:43.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.081+0000 7fdd037ff700 1 Processor -- start 2023-08-25T04:42:43.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 -- start start 2023-08-25T04:42:43.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:43.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:43.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 0x7fdcfc19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:43.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1a0410 con 0x7fdcfc100930 2023-08-25T04:42:43.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1a0550 con 0x7fdcfc101b20 2023-08-25T04:42:43.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd037ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1a06e0 con 0x7fdcfc1024c0 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd0159b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd0159b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48232/0 (socket says 172.21.15.145:48232) 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd0159b700 1 -- 172.21.15.145:0/4019976227 learned_addr learned my addr 172.21.15.145:0/4019976227 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd00d9a700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.082+0000 7fdd01d9c700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 0x7fdcfc19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:43.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 -- 172.21.15.145:0/4019976227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 msgr2=0x7fdcfc19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:43.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 0x7fdcfc19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 -- 172.21.15.145:0/4019976227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 msgr2=0x7fdcfc198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:43.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 -- 172.21.15.145:0/4019976227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec009520 con 0x7fdcfc100930 2023-08-25T04:42:43.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd0159b700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800e9f0 tx=0x7fdcf800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8004220 con 0x7fdcfc100930 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf8004380 con 0x7fdcfc100930 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc1a09c0 con 0x7fdcfc100930 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8003c00 con 0x7fdcfc100930 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.083+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc1a0ed0 con 0x7fdcfc100930 2023-08-25T04:42:43.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.084+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc192f80 con 0x7fdcfc100930 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.086+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fdcf800c7c0 con 0x7fdcfc100930 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.088+0000 7fdcf27fc700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 0x7fdce80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.088+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcf8014070 con 0x7fdcfc100930 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.088+0000 7fdd00d9a700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 0x7fdce80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.089+0000 7fdd00d9a700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 0x7fdce80a2460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00fa40 tx=0x7fdcec019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:43.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.092+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fdcf8095360 con 0x7fdcfc100930 2023-08-25T04:42:43.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:42 smithi183 ceph-mon[131414]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:43.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.347+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcfc10c190 con 0x7fdcfc100930 2023-08-25T04:42:43.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.347+0000 7fdcf27fc700 1 -- 172.21.15.145:0/4019976227 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9266 (secure 0 0 0) 0x7fdcf8098980 con 0x7fdcfc100930 2023-08-25T04:42:43.348 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:43.348 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-08-25T04:42:43.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 msgr2=0x7fdce80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 0x7fdce80a2460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00fa40 tx=0x7fdcec019040 comp rx=0 tx=0).stop 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 msgr2=0x7fdcfc198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800e9f0 tx=0x7fdcf800ed00 comp rx=0 tx=0).stop 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 shutdown_connections 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdce809ffa0 0x7fdce80a2460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcfc1024c0 0x7fdcfc19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdcfc100930 0x7fdcfc198890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 --2- 172.21.15.145:0/4019976227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcfc101b20 0x7fdcfc198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:43.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 >> 172.21.15.145:0/4019976227 conn(0x7fdcfc0fbe60 msgr2=0x7fdcfc107480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:43.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 shutdown_connections 2023-08-25T04:42:43.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:43.350+0000 7fdd037ff700 1 -- 172.21.15.145:0/4019976227 wait complete. 2023-08-25T04:42:43.352 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 12 2023-08-25T04:42:43.411 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 04:42:42 smithi162 systemd[1]: Started Ceph mds.b for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T04:42:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4019976227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:boot 2023-08-25T04:42:44.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: fsmap 12 up:standby 2023-08-25T04:42:44.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:43 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:42:44.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4019976227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:boot 2023-08-25T04:42:44.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: fsmap 12 up:standby 2023-08-25T04:42:44.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:43 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:42:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4019976227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:boot 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: fsmap 12 up:standby 2023-08-25T04:42:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:43 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:42:45.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:44 smithi145 ceph-mon[97681]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:45.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:45.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:44 smithi162 ceph-mon[130941]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:45.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:44 smithi183 ceph-mon[131414]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:45.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 -- 172.21.15.145:0/1319347675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 msgr2=0x7fc064108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:45.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/1319347675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064108d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b0d0 tx=0x7fc05c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 -- 172.21.15.145:0/1319347675 shutdown_connections 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/1319347675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064108d70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/1319347675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 0x7fc064100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/1319347675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 -- 172.21.15.145:0/1319347675 >> 172.21.15.145:0/1319347675 conn(0x7fc0640fbe60 msgr2=0x7fc0640fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 -- 172.21.15.145:0/1319347675 shutdown_connections 2023-08-25T04:42:45.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.592+0000 7fc06a5ff700 1 -- 172.21.15.145:0/1319347675 wait complete. 2023-08-25T04:42:45.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.593+0000 7fc06a5ff700 1 Processor -- start 2023-08-25T04:42:45.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.593+0000 7fc06a5ff700 1 -- start start 2023-08-25T04:42:45.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.593+0000 7fc06a5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 0x7fc0640746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.593+0000 7fc06a5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc06a5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc06a5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640739f0 con 0x7fc0641024c0 2023-08-25T04:42:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc06a5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc064073b60 con 0x7fc064100930 2023-08-25T04:42:45.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc06a5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0641a06c0 con 0x7fc064101b20 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc068b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc063fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 0x7fc0640746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc068b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48236/0 (socket says 172.21.15.145:48236) 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc068b9c700 1 -- 172.21.15.145:0/3316643210 learned_addr learned my addr 172.21.15.145:0/3316643210 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 -- 172.21.15.145:0/3316643210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 msgr2=0x7fc0640746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:45.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 0x7fc0640746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 -- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 msgr2=0x7fc064073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 -- 172.21.15.145:0/3316643210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058009520 con 0x7fc064101b20 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc068b9c700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.594+0000 7fc0637fe700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064072d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e9f0 tx=0x7fc05800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:45.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.595+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05800cb70 con 0x7fc064101b20 2023-08-25T04:42:45.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.595+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c009d00 con 0x7fc064101b20 2023-08-25T04:42:45.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.595+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc05800ee90 con 0x7fc064101b20 2023-08-25T04:42:45.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.595+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058003cf0 con 0x7fc064101b20 2023-08-25T04:42:45.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.595+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0641a0d20 con 0x7fc064101b20 2023-08-25T04:42:45.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.596+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc064107e00 con 0x7fc064101b20 2023-08-25T04:42:45.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.598+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc05800ccd0 con 0x7fc064101b20 2023-08-25T04:42:45.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.599+0000 7fc0617fa700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 0x7fc04c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:45.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.599+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc058014070 con 0x7fc064101b20 2023-08-25T04:42:45.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.599+0000 7fc063fff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 0x7fc04c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:45.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.600+0000 7fc063fff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 0x7fc04c0a2460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc064101980 tx=0x7fc054009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:45.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.603+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc058095250 con 0x7fc064101b20 2023-08-25T04:42:45.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.855+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc06410c150 con 0x7fc064101b20 2023-08-25T04:42:45.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.856+0000 7fc0617fa700 1 -- 172.21.15.145:0/3316643210 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10061 (secure 0 0 0) 0x7fc058098870 con 0x7fc064101b20 2023-08-25T04:42:45.857 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:45.857 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-08-25T04:42:45.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 msgr2=0x7fc04c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:45.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 0x7fc04c0a2460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc064101980 tx=0x7fc054009260 comp rx=0 tx=0).stop 2023-08-25T04:42:45.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 msgr2=0x7fc064072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:45.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064072d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e9f0 tx=0x7fc05800ed00 comp rx=0 tx=0).stop 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 shutdown_connections 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc04c09ffa0 0x7fc04c0a2460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0641024c0 0x7fc064073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc064100930 0x7fc0640746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 --2- 172.21.15.145:0/3316643210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc064101b20 0x7fc064072d10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 >> 172.21.15.145:0/3316643210 conn(0x7fc0640fbe60 msgr2=0x7fc064104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 shutdown_connections 2023-08-25T04:42:45.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:45.859+0000 7fc06a5ff700 1 -- 172.21.15.145:0/3316643210 wait complete. 2023-08-25T04:42:45.863 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 13 2023-08-25T04:42:45.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:42:46.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 -- 172.21.15.145:0/3733916166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 msgr2=0x7f26a4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 --2- 172.21.15.145:0/3733916166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a4100d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2698009990 tx=0x7f2698009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 -- 172.21.15.145:0/3733916166 shutdown_connections 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 --2- 172.21.15.145:0/3733916166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a41024c0 0x7f26a4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 --2- 172.21.15.145:0/3733916166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a4100d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 --2- 172.21.15.145:0/3733916166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a4101b20 0x7f26a4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 -- 172.21.15.145:0/3733916166 >> 172.21.15.145:0/3733916166 conn(0x7f26a40fbe60 msgr2=0x7f26a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 -- 172.21.15.145:0/3733916166 shutdown_connections 2023-08-25T04:42:46.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.092+0000 7f26aabff700 1 -- 172.21.15.145:0/3733916166 wait complete. 2023-08-25T04:42:46.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.093+0000 7f26aabff700 1 Processor -- start 2023-08-25T04:42:46.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.093+0000 7f26aabff700 1 -- start start 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.093+0000 7f26aabff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.093+0000 7f26aabff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 0x7f26a4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26aabff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a41024c0 0x7f26a419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26aabff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41a0360 con 0x7f26a41024c0 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26aabff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41a04a0 con 0x7f26a4101b20 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26aabff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41a0630 con 0x7f26a4100930 2023-08-25T04:42:46.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50104/0 (socket says 172.21.15.145:50104) 2023-08-25T04:42:46.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 -- 172.21.15.145:0/1224039951 learned_addr learned my addr 172.21.15.145:0/1224039951 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:46.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a3fff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 0x7f26a4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a919c700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a41024c0 0x7f26a419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 -- 172.21.15.145:0/1224039951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 msgr2=0x7f26a4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 0x7f26a4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 -- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a41024c0 msgr2=0x7f26a419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a41024c0 0x7f26a419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 -- 172.21.15.145:0/1224039951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009520 con 0x7f26a4100930 2023-08-25T04:42:46.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a3fff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 0x7f26a4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.094+0000 7f26a899b700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2698005720 tx=0x7f2698009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.095+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269801d070 con 0x7f26a4100930 2023-08-25T04:42:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.095+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2698021410 con 0x7f26a4100930 2023-08-25T04:42:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.095+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26980095f0 con 0x7f26a4100930 2023-08-25T04:42:46.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.095+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698017670 con 0x7f26a4100930 2023-08-25T04:42:46.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.095+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a41a0b40 con 0x7f26a4100930 2023-08-25T04:42:46.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.096+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a410c190 con 0x7f26a4100930 2023-08-25T04:42:46.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.098+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f2698021580 con 0x7f26a4100930 2023-08-25T04:42:46.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.099+0000 7f26a1ffb700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 0x7f26940a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.099+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26980cc1a0 con 0x7f26a4100930 2023-08-25T04:42:46.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.100+0000 7f26a3fff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 0x7f26940a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.100+0000 7f26a3fff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 0x7f26940a2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2690009fd0 tx=0x7f2690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:46.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.101+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f26980963c0 con 0x7f26a4100930 2023-08-25T04:42:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.322+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26a41a0d60 con 0x7f26a4100930 2023-08-25T04:42:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.323+0000 7f26a1ffb700 1 -- 172.21.15.145:0/1224039951 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18468 (secure 0 0 0) 0x7f26980999e0 con 0x7f26a4100930 2023-08-25T04:42:46.324 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:46.325 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":79,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:04.548924+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:46.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 msgr2=0x7f26940a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 0x7f26940a2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2690009fd0 tx=0x7f2690009260 comp rx=0 tx=0).stop 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 msgr2=0x7f26a41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2698005720 tx=0x7f2698009580 comp rx=0 tx=0).stop 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 shutdown_connections 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f269409ffa0 0x7f26940a2460 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26a41024c0 0x7f26a419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26a4100930 0x7f26a41987e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 --2- 172.21.15.145:0/1224039951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26a4101b20 0x7f26a4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 >> 172.21.15.145:0/1224039951 conn(0x7f26a40fbe60 msgr2=0x7f26a41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 shutdown_connections 2023-08-25T04:42:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.326+0000 7f26aabff700 1 -- 172.21.15.145:0/1224039951 wait complete. 2023-08-25T04:42:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:45 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3316643210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:46.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T04:42:46.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:42:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:45 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3316643210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:46.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:45 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3316643210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:46.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 -- 172.21.15.145:0/559540178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac10cd70 msgr2=0x7fc1ac10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 --2- 172.21.15.145:0/559540178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac10f160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b0a0 tx=0x7fc1a400b3b0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 -- 172.21.15.145:0/559540178 shutdown_connections 2023-08-25T04:42:46.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 --2- 172.21.15.145:0/559540178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac10f160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 --2- 172.21.15.145:0/559540178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac074960 0x7fc1ac072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 --2- 172.21.15.145:0/559540178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac073380 0x7fc1ac073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.559+0000 7fc1b259d700 1 -- 172.21.15.145:0/559540178 >> 172.21.15.145:0/559540178 conn(0x7fc1ac0fbd30 msgr2=0x7fc1ac0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:46.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.560+0000 7fc1b259d700 1 -- 172.21.15.145:0/559540178 shutdown_connections 2023-08-25T04:42:46.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.560+0000 7fc1b259d700 1 -- 172.21.15.145:0/559540178 wait complete. 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.560+0000 7fc1b259d700 1 Processor -- start 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 -- start start 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac074960 0x7fc1ac19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a45b0 con 0x7fc1ac10cd70 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a46f0 con 0x7fc1ac074960 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b259d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1a4880 con 0x7fc1ac073380 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b1d9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b159b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b1d9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48272/0 (socket says 172.21.15.145:48272) 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b159b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50132/0 (socket says 172.21.15.145:50132) 2023-08-25T04:42:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b1d9c700 1 -- 172.21.15.145:0/2029639004 learned_addr learned my addr 172.21.15.145:0/2029639004 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.561+0000 7fc1b0d9a700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac074960 0x7fc1ac19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 -- 172.21.15.145:0/2029639004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 msgr2=0x7fc1ac19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 -- 172.21.15.145:0/2029639004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac074960 msgr2=0x7fc1ac19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac074960 0x7fc1ac19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 -- 172.21.15.145:0/2029639004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a4009d00 con 0x7fc1ac10cd70 2023-08-25T04:42:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b159b700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:46.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b1d9c700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b780 tx=0x7fc1a400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:46.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400e040 con 0x7fc1ac10cd70 2023-08-25T04:42:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a4004c50 con 0x7fc1ac10cd70 2023-08-25T04:42:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac1a4b00 con 0x7fc1ac10cd70 2023-08-25T04:42:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4005280 con 0x7fc1ac10cd70 2023-08-25T04:42:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.562+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1a4f80 con 0x7fc1ac10cd70 2023-08-25T04:42:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.564+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac1028c0 con 0x7fc1ac10cd70 2023-08-25T04:42:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.565+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc1a401d470 con 0x7fc1ac10cd70 2023-08-25T04:42:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.566+0000 7fc1a27fc700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 0x7fc1980a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.567+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1a40cc300 con 0x7fc1ac10cd70 2023-08-25T04:42:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.567+0000 7fc1b159b700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 0x7fc1980a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.567+0000 7fc1b159b700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 0x7fc1980a24b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8011490 tx=0x7fc1a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.571+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc1a4096520 con 0x7fc1ac10cd70 2023-08-25T04:42:46.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.804+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1ac02cd70 con 0x7fc1ac10cd70 2023-08-25T04:42:46.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.804+0000 7fc1a27fc700 1 -- 172.21.15.145:0/2029639004 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1a4099b40 con 0x7fc1ac10cd70 2023-08-25T04:42:46.806 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 msgr2=0x7fc1980a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 0x7fc1980a24b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8011490 tx=0x7fc1a8009260 comp rx=0 tx=0).stop 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 msgr2=0x7fc1ac1a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b780 tx=0x7fc1a400bab0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 shutdown_connections 2023-08-25T04:42:46.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc19809fff0 0x7fc1980a24b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1ac10cd70 0x7fc1ac1a3fe0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1ac073380 0x7fc1ac19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 --2- 172.21.15.145:0/2029639004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ac074960 0x7fc1ac19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 >> 172.21.15.145:0/2029639004 conn(0x7fc1ac0fbd30 msgr2=0x7fc1ac10ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 shutdown_connections 2023-08-25T04:42:46.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:46.807+0000 7fc1b259d700 1 -- 172.21.15.145:0/2029639004 wait complete. 2023-08-25T04:42:46.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T04:42:47.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 -- 172.21.15.145:0/477454183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 msgr2=0x7f68080ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 --2- 172.21.15.145:0/477454183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f68080ffc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009990 tx=0x7f67f8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 -- 172.21.15.145:0/477454183 shutdown_connections 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 --2- 172.21.15.145:0/477454183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 0x7f6808107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 --2- 172.21.15.145:0/477454183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f68080ffc30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 --2- 172.21.15.145:0/477454183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 0x7f6808100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 -- 172.21.15.145:0/477454183 >> 172.21.15.145:0/477454183 conn(0x7f68080fad80 msgr2=0x7f68080fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:47.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 -- 172.21.15.145:0/477454183 shutdown_connections 2023-08-25T04:42:47.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.034+0000 7f680dfff700 1 -- 172.21.15.145:0/477454183 wait complete. 2023-08-25T04:42:47.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.035+0000 7f680dfff700 1 Processor -- start 2023-08-25T04:42:47.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 -- start start 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 0x7f6808198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 0x7f680819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081a03f0 con 0x7f6808100a00 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081a0530 con 0x7f68081013a0 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f680dfff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081a06c0 con 0x7f68080ff810 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f68077fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f68077fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50162/0 (socket says 172.21.15.145:50162) 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f68077fe700 1 -- 172.21.15.145:0/575247986 learned_addr learned my addr 172.21.15.145:0/575247986 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f6807fff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 0x7f680819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.036+0000 7f6806ffd700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 0x7f6808198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 -- 172.21.15.145:0/575247986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 msgr2=0x7f680819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 0x7f680819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 -- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 msgr2=0x7f6808198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 0x7f6808198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 -- 172.21.15.145:0/575247986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80095f0 con 0x7f68080ff810 2023-08-25T04:42:47.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f68077fe700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005720 tx=0x7f67f8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:47.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f801d070 con 0x7f68080ff810 2023-08-25T04:42:47.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f8021410 con 0x7f68080ff810 2023-08-25T04:42:47.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68081a0940 con 0x7f68080ff810 2023-08-25T04:42:47.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8017650 con 0x7f68080ff810 2023-08-25T04:42:47.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.037+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081a0dc0 con 0x7f68080ff810 2023-08-25T04:42:47.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.039+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680810b070 con 0x7f68080ff810 2023-08-25T04:42:47.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.040+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f67f8021580 con 0x7f68080ff810 2023-08-25T04:42:47.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.041+0000 7f6804ff9700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 0x7f67f00a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.042+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67f80cbf60 con 0x7f68080ff810 2023-08-25T04:42:47.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.042+0000 7f6806ffd700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 0x7f67f00a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.042+0000 7f6806ffd700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 0x7f67f00a24e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009c60 tx=0x7f67fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:47.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.045+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f67f80962a0 con 0x7f68080ff810 2023-08-25T04:42:47.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1224039951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2029639004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:47.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:46 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.268+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f68081a1240 con 0x7f68080ff810 2023-08-25T04:42:47.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.269+0000 7f6804ff9700 1 -- 172.21.15.145:0/575247986 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f67f8026060 con 0x7f68080ff810 2023-08-25T04:42:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:47.269 INFO:teuthology.orchestra.run.smithi145.stdout:[ 2023-08-25T04:42:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T04:42:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 1, 2023-08-25T04:42:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": ".mgr" 2023-08-25T04:42:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:42:47.270 INFO:teuthology.orchestra.run.smithi145.stdout:] 2023-08-25T04:42:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 msgr2=0x7f67f00a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 0x7f67f00a24e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009c60 tx=0x7f67fc009260 comp rx=0 tx=0).stop 2023-08-25T04:42:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 msgr2=0x7f6808198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005720 tx=0x7f67f8009580 comp rx=0 tx=0).stop 2023-08-25T04:42:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 shutdown_connections 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f67f00a0020 0x7f67f00a24e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68081013a0 0x7f680819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080ff810 0x7f6808198870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 --2- 172.21.15.145:0/575247986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6808100a00 0x7f6808198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 >> 172.21.15.145:0/575247986 conn(0x7f68080fad80 msgr2=0x7f6808068d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 shutdown_connections 2023-08-25T04:42:47.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.271+0000 7f680dfff700 1 -- 172.21.15.145:0/575247986 wait complete. 2023-08-25T04:42:47.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T04:42:47.291 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T04:42:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1224039951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:47.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:47.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2029639004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:47.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:46 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1224039951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2029639004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:42:47.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:46 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:42:47.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.503+0000 7fa17c7ff700 1 -- 172.21.15.145:0/2738041923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1741024a0 msgr2=0x7fa174108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.503+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/2738041923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1741024a0 0x7fa174108d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0099c0 tx=0x7fa16c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 -- 172.21.15.145:0/2738041923 shutdown_connections 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/2738041923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1741024a0 0x7fa174108d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/2738041923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa174100910 0x7fa174100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/2738041923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174101b00 0x7fa174101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 -- 172.21.15.145:0/2738041923 >> 172.21.15.145:0/2738041923 conn(0x7fa1740fbe60 msgr2=0x7fa1740fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:47.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 -- 172.21.15.145:0/2738041923 shutdown_connections 2023-08-25T04:42:47.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.504+0000 7fa17c7ff700 1 -- 172.21.15.145:0/2738041923 wait complete. 2023-08-25T04:42:47.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 Processor -- start 2023-08-25T04:42:47.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 -- start start 2023-08-25T04:42:47.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174100910 0x7fa174198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741a03e0 con 0x7fa174101b00 2023-08-25T04:42:47.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741a0520 con 0x7fa174100910 2023-08-25T04:42:47.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17c7ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741a06b0 con 0x7fa1741024a0 2023-08-25T04:42:47.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa179d9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17ad9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa179d9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48312/0 (socket says 172.21.15.145:48312) 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa179d9a700 1 -- 172.21.15.145:0/1789817778 learned_addr learned my addr 172.21.15.145:0/1789817778 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17ad9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50178/0 (socket says 172.21.15.145:50178) 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.505+0000 7fa17a59b700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174100910 0x7fa174198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 -- 172.21.15.145:0/1789817778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174100910 msgr2=0x7fa174198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174100910 0x7fa174198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 -- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 msgr2=0x7fa174198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 -- 172.21.15.145:0/1789817778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170009c40 con 0x7fa1741024a0 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa179d9a700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17ad9c700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01d070 con 0x7fa1741024a0 2023-08-25T04:42:47.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c0095f0 con 0x7fa1741024a0 2023-08-25T04:42:47.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16c005050 con 0x7fa1741024a0 2023-08-25T04:42:47.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.506+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0049b0 con 0x7fa1741024a0 2023-08-25T04:42:47.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.507+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741a0bd0 con 0x7fa1741024a0 2023-08-25T04:42:47.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.507+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa174192f70 con 0x7fa1741024a0 2023-08-25T04:42:47.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.509+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa16c0054c0 con 0x7fa1741024a0 2023-08-25T04:42:47.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.510+0000 7fa16b7fe700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 0x7fa1600a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:47.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.511+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa16c0cbb80 con 0x7fa1741024a0 2023-08-25T04:42:47.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.511+0000 7fa17a59b700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 0x7fa1600a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:47.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.511+0000 7fa17a59b700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 0x7fa1600a2410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa174101960 tx=0x7fa17000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:47.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.512+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa16c095da0 con 0x7fa1741024a0 2023-08-25T04:42:47.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:47.743+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa17410c180 con 0x7fa1741024a0 2023-08-25T04:42:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.310+0000 7fa16b7fe700 1 -- 172.21.15.145:0/1789817778 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa16c0993c0 con 0x7fa1741024a0 2023-08-25T04:42:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 msgr2=0x7fa1600a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 0x7fa1600a2410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa174101960 tx=0x7fa17000b040 comp rx=0 tx=0).stop 2023-08-25T04:42:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 msgr2=0x7fa17419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c00f690 comp rx=0 tx=0).stop 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 shutdown_connections 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa16009ff50 0x7fa1600a2410 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1741024a0 0x7fa17419fe10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa174100910 0x7fa174198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 --2- 172.21.15.145:0/1789817778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa174101b00 0x7fa174198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 >> 172.21.15.145:0/1789817778 conn(0x7fa1740fbe60 msgr2=0x7fa1741073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 shutdown_connections 2023-08-25T04:42:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.315+0000 7fa17c7ff700 1 -- 172.21.15.145:0/1789817778 wait complete. 2023-08-25T04:42:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' created 2023-08-25T04:42:48.335 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-25T04:42:48.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/4288354262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8073380 msgr2=0x7fb2e8073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:48.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/4288354262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8073380 0x7fb2e8073800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009990 tx=0x7fb2dc009ca0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/4288354262 shutdown_connections 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/4288354262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 0x7fb2e810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/4288354262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8074960 0x7fb2e8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/4288354262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8073380 0x7fb2e8073800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/4288354262 >> 172.21.15.145:0/4288354262 conn(0x7fb2e80fbea0 msgr2=0x7fb2e80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/4288354262 shutdown_connections 2023-08-25T04:42:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.553+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/4288354262 wait complete. 2023-08-25T04:42:48.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.554+0000 7fb2ee5ff700 1 Processor -- start 2023-08-25T04:42:48.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 -- start start 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 0x7fb2e8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 0x7fb2e819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81a0410 con 0x7fb2e8073380 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81a0550 con 0x7fb2e81087f0 2023-08-25T04:42:48.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ee5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81a06e0 con 0x7fb2e8074960 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2e7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2e7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48332/0 (socket says 172.21.15.145:48332) 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2e7fff700 1 -- 172.21.15.145:0/1150442811 learned_addr learned my addr 172.21.15.145:0/1150442811 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2e77fe700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 0x7fb2e8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.555+0000 7fb2ecb9c700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 0x7fb2e819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:48.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 -- 172.21.15.145:0/1150442811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 msgr2=0x7fb2e8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:48.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 0x7fb2e8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 -- 172.21.15.145:0/1150442811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 msgr2=0x7fb2e819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:48.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 0x7fb2e819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:48.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 -- 172.21.15.145:0/1150442811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0095f0 con 0x7fb2e8073380 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e77fe700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 0x7fb2e8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e7fff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9a0 tx=0x7fb2d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8009790 con 0x7fb2e8073380 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e81a09c0 con 0x7fb2e8073380 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d8010430 con 0x7fb2e8073380 2023-08-25T04:42:48.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8018680 con 0x7fb2e8073380 2023-08-25T04:42:48.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.556+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e81a0f20 con 0x7fb2e8073380 2023-08-25T04:42:48.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.557+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e8106ea0 con 0x7fb2e8073380 2023-08-25T04:42:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.560+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fb2d80105a0 con 0x7fb2e8073380 2023-08-25T04:42:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.561+0000 7fb2e57fa700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 0x7fb2d00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.562+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fb2d8014070 con 0x7fb2e8073380 2023-08-25T04:42:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.562+0000 7fb2e77fe700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 0x7fb2d00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:48.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.563+0000 7fb2e77fe700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 0x7fb2d00a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00b2e0 tx=0x7fb2dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:48.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.564+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb2d8095360 con 0x7fb2e8073380 2023-08-25T04:42:48.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:48 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/575247986' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T04:42:48.581 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:48 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1789817778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:48.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:48 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/575247986' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T04:42:48.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:48 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1789817778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:48.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:48 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/575247986' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-25T04:42:48.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:48 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1789817778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:48.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:48.780+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb2e810e5e0 con 0x7fb2e8073380 2023-08-25T04:42:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.309+0000 7fb2e57fa700 1 -- 172.21.15.145:0/1150442811 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb2d8098980 con 0x7fb2e8073380 2023-08-25T04:42:49.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.315+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 msgr2=0x7fb2d00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:49.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.315+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 0x7fb2d00a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00b2e0 tx=0x7fb2dc005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.315+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 msgr2=0x7fb2e8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:49.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.315+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9a0 tx=0x7fb2d800ecb0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 shutdown_connections 2023-08-25T04:42:49.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb2d009ff50 0x7fb2d00a2410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2e81087f0 0x7fb2e819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e8073380 0x7fb2e8198890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 --2- 172.21.15.145:0/1150442811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2e8074960 0x7fb2e8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.319+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 >> 172.21.15.145:0/1150442811 conn(0x7fb2e80fbea0 msgr2=0x7fb2e80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:49.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.322+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 shutdown_connections 2023-08-25T04:42:49.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.323+0000 7fb2ee5ff700 1 -- 172.21.15.145:0/1150442811 wait complete. 2023-08-25T04:42:49.330 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' created 2023-08-25T04:42:49.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T04:42:49.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 -- 172.21.15.145:0/2591950975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510100930 msgr2=0x7f3510100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:49.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 --2- 172.21.15.145:0/2591950975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510100930 0x7f3510100d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3500009920 tx=0x7f3500009c30 comp rx=0 tx=0).stop 2023-08-25T04:42:49.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 -- 172.21.15.145:0/2591950975 shutdown_connections 2023-08-25T04:42:49.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 --2- 172.21.15.145:0/2591950975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 --2- 172.21.15.145:0/2591950975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510100930 0x7f3510100d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 --2- 172.21.15.145:0/2591950975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510101b20 0x7f3510101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 -- 172.21.15.145:0/2591950975 >> 172.21.15.145:0/2591950975 conn(0x7f35100fbea0 msgr2=0x7f35100fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:49.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 -- 172.21.15.145:0/2591950975 shutdown_connections 2023-08-25T04:42:49.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.562+0000 7f35149ff700 1 -- 172.21.15.145:0/2591950975 wait complete. 2023-08-25T04:42:49.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.563+0000 7f35149ff700 1 Processor -- start 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 -- start start 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 0x7f35100746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510101b20 0x7f3510072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100739f0 con 0x7f35101024c0 2023-08-25T04:42:49.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510073b60 con 0x7f3510101b20 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f35149ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35101a0760 con 0x7f3510100930 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f350ed9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f350e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 0x7f35100746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f350ed9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48348/0 (socket says 172.21.15.145:48348) 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.564+0000 7f350ed9d700 1 -- 172.21.15.145:0/3245581488 learned_addr learned my addr 172.21.15.145:0/3245581488 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 -- 172.21.15.145:0/3245581488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510101b20 msgr2=0x7f3510072d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:42:49.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510101b20 0x7f3510072d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 -- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 msgr2=0x7f3510073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 -- 172.21.15.145:0/3245581488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009520 con 0x7f3510100930 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350ed9d700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f350e59c700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 0x7f35100746d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3500000c00 tx=0x7f350000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350001c070 con 0x7f3510100930 2023-08-25T04:42:49.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35101a09e0 con 0x7f3510100930 2023-08-25T04:42:49.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35101a0e00 con 0x7f3510100930 2023-08-25T04:42:49.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350000f7d0 con 0x7f3510100930 2023-08-25T04:42:49.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.565+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000038e0 con 0x7f3510100930 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.567+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351010c190 con 0x7f3510100930 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.568+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3500020030 con 0x7f3510100930 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.569+0000 7f34ff7fe700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 0x7f34f80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.569+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f35000cc080 con 0x7f3510100930 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.570+0000 7f350dd9b700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 0x7f34f80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:49.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.570+0000 7f350dd9b700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 0x7f34f80a2410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3504005fd0 tx=0x7f3504009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:49.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.573+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3500095fb0 con 0x7f3510100930 2023-08-25T04:42:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:49 smithi145 ceph-mon[97681]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:49.584 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:49 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:49.584 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:49 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T04:42:49.585 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:49 smithi145 ceph-mon[97681]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T04:42:49.585 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:49 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-25T04:42:49.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:49 smithi162 ceph-mon[130941]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:49 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:49 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T04:42:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:49 smithi162 ceph-mon[130941]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T04:42:49.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:49 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-25T04:42:49.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:49 smithi183 ceph-mon[131414]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:49.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:49 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-25T04:42:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:49 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-25T04:42:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:49 smithi183 ceph-mon[131414]: osdmap e80: 12 total, 12 up, 12 in 2023-08-25T04:42:49.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:49 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-25T04:42:49.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:49.860+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f351010c320 con 0x7f3510100930 2023-08-25T04:42:50.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: pgmap v259: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3245581488' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:50 smithi145 ceph-mon[97681]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-08-25T04:42:50.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-25T04:42:50.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T04:42:50.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: pgmap v259: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3245581488' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:50 smithi162 ceph-mon[130941]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-08-25T04:42:50.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1150442811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: osdmap e81: 12 total, 12 up, 12 in 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: pgmap v259: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:50.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3245581488' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-25T04:42:50.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:50 smithi183 ceph-mon[131414]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-08-25T04:42:50.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.820+0000 7f34ff7fe700 1 -- 172.21.15.145:0/3245581488 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f351010c320 con 0x7f3510100930 2023-08-25T04:42:50.828 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.825+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 msgr2=0x7f34f80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.825+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 0x7f34f80a2410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3504005fd0 tx=0x7f3504009310 comp rx=0 tx=0).stop 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.825+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 msgr2=0x7f35100746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.825+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 0x7f35100746d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3500000c00 tx=0x7f350000b790 comp rx=0 tx=0).stop 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 shutdown_connections 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34f809ff50 0x7f34f80a2410 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35101024c0 0x7f3510073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:50.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3510100930 0x7f35100746d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:50.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 --2- 172.21.15.145:0/3245581488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3510101b20 0x7f3510072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:50.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.826+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 >> 172.21.15.145:0/3245581488 conn(0x7f35100fbea0 msgr2=0x7f35100fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:50.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.827+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 shutdown_connections 2023-08-25T04:42:50.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:50.827+0000 7f35149ff700 1 -- 172.21.15.145:0/3245581488 wait complete. 2023-08-25T04:42:50.830 INFO:teuthology.orchestra.run.smithi145.stderr:new fs with metadata pool 2 and data pool 3 2023-08-25T04:42:50.848 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-25T04:42:50.849 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-25T04:42:51.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 -- 172.21.15.145:0/3808338812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de4100930 msgr2=0x7f0de4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3808338812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de4100930 0x7f0de4100d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40099c0 tx=0x7f0dd4009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 -- 172.21.15.145:0/3808338812 shutdown_connections 2023-08-25T04:42:51.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3808338812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de41024c0 0x7f0de4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3808338812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de4100930 0x7f0de4100d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3808338812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4101b20 0x7f0de4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 -- 172.21.15.145:0/3808338812 >> 172.21.15.145:0/3808338812 conn(0x7f0de40fbea0 msgr2=0x7f0de40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:51.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 -- 172.21.15.145:0/3808338812 shutdown_connections 2023-08-25T04:42:51.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.052+0000 7f0de95ff700 1 -- 172.21.15.145:0/3808338812 wait complete. 2023-08-25T04:42:51.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 Processor -- start 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 -- start start 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 0x7f0de4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 0x7f0de4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 0x7f0de419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de41a03b0 con 0x7f0de41024c0 2023-08-25T04:42:51.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de41a04f0 con 0x7f0de4100930 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.053+0000 7f0de95ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de41a0680 con 0x7f0de4101b20 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 0x7f0de4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 0x7f0de419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0ddbfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 0x7f0de4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 0x7f0de4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57222/0 (socket says 172.21.15.145:57222) 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de2ffd700 1 -- 172.21.15.145:0/3870397131 learned_addr learned my addr 172.21.15.145:0/3870397131 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:51.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 -- 172.21.15.145:0/3870397131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 msgr2=0x7f0de4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 0x7f0de4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 -- 172.21.15.145:0/3870397131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 msgr2=0x7f0de4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 0x7f0de4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 -- 172.21.15.145:0/3870397131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40095f0 con 0x7f0de41024c0 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0ddbfff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 0x7f0de4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de37fe700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 0x7f0de419fde0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00efc0 tx=0x7f0ddc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:51.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.054+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc018070 con 0x7f0de41024c0 2023-08-25T04:42:51.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.055+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ddc00f040 con 0x7f0de41024c0 2023-08-25T04:42:51.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.055+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00cd40 con 0x7f0de41024c0 2023-08-25T04:42:51.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.055+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de41a0900 con 0x7f0de41024c0 2023-08-25T04:42:51.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.055+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de41a0dc0 con 0x7f0de41024c0 2023-08-25T04:42:51.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.056+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de4192f80 con 0x7f0de41024c0 2023-08-25T04:42:51.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.058+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f0ddc003a00 con 0x7f0de41024c0 2023-08-25T04:42:51.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.059+0000 7f0de0ff9700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 0x7f0dd00a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.059+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ddc0cb640 con 0x7f0de41024c0 2023-08-25T04:42:51.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.060+0000 7f0de2ffd700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 0x7f0dd00a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.060+0000 7f0de2ffd700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 0x7f0dd00a2510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000c00 tx=0x7f0dd4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:51.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.062+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0ddc095510 con 0x7f0de41024c0 2023-08-25T04:42:51.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.276+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0de410c190 con 0x7f0de41024c0 2023-08-25T04:42:51.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.316+0000 7f0de0ff9700 1 -- 172.21.15.145:0/3870397131 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0ddc098b30 con 0x7f0de41024c0 2023-08-25T04:42:51.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 msgr2=0x7f0dd00a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 0x7f0dd00a2510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000c00 tx=0x7f0dd4005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 msgr2=0x7f0de419fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 0x7f0de419fde0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00efc0 tx=0x7f0ddc00c3a0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 shutdown_connections 2023-08-25T04:42:51.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f0dd00a0050 0x7f0dd00a2510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0de41024c0 0x7f0de419fde0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de4100930 0x7f0de4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 --2- 172.21.15.145:0/3870397131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0de4101b20 0x7f0de4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 >> 172.21.15.145:0/3870397131 conn(0x7f0de40fbea0 msgr2=0x7f0de41073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:51.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 shutdown_connections 2023-08-25T04:42:51.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.319+0000 7f0de95ff700 1 -- 172.21.15.145:0/3870397131 wait complete. 2023-08-25T04:42:51.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T04:42:51.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 -- 172.21.15.145:0/3208706872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 msgr2=0x7f2688101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3208706872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688101f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f267c0099c0 tx=0x7f267c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 -- 172.21.15.145:0/3208706872 shutdown_connections 2023-08-25T04:42:51.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3208706872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 0x7f2688108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3208706872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 0x7f2688100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3208706872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688101f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 -- 172.21.15.145:0/3208706872 >> 172.21.15.145:0/3208706872 conn(0x7f26880fbe60 msgr2=0x7f26880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:51.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 -- 172.21.15.145:0/3208706872 shutdown_connections 2023-08-25T04:42:51.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.543+0000 7f268d1ff700 1 -- 172.21.15.145:0/3208706872 wait complete. 2023-08-25T04:42:51.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.544+0000 7f268d1ff700 1 Processor -- start 2023-08-25T04:42:51.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.544+0000 7f268d1ff700 1 -- start start 2023-08-25T04:42:51.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 0x7f2688198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 0x7f268819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881a0390 con 0x7f2688101b20 2023-08-25T04:42:51.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881a04d0 con 0x7f2688100930 2023-08-25T04:42:51.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268d1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881a0660 con 0x7f26881024c0 2023-08-25T04:42:51.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:48398/0 (socket says 172.21.15.145:48398) 2023-08-25T04:42:51.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 -- 172.21.15.145:0/3248922454 learned_addr learned my addr 172.21.15.145:0/3248922454 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:51.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f2686d9d700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 0x7f2688198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268759e700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 0x7f268819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 -- 172.21.15.145:0/3248922454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 msgr2=0x7f268819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 0x7f268819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 -- 172.21.15.145:0/3248922454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 msgr2=0x7f2688198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 0x7f2688198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.545+0000 7f268659c700 1 -- 172.21.15.145:0/3248922454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0095f0 con 0x7f2688101b20 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f268659c700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c01d070 con 0x7f2688101b20 2023-08-25T04:42:51.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f267c0054f0 con 0x7f2688101b20 2023-08-25T04:42:51.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26881a08e0 con 0x7f2688101b20 2023-08-25T04:42:51.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c004b00 con 0x7f2688101b20 2023-08-25T04:42:51.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.546+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26881a0d60 con 0x7f2688101b20 2023-08-25T04:42:51.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.547+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688192f80 con 0x7f2688101b20 2023-08-25T04:42:51.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.549+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f267c00fb80 con 0x7f2688101b20 2023-08-25T04:42:51.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.550+0000 7f266ffff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 0x7f26700a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:51.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.551+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f267c0cbf70 con 0x7f2688101b20 2023-08-25T04:42:51.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.551+0000 7f2686d9d700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 0x7f26700a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:51.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.551+0000 7f2686d9d700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 0x7f26700a2530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2688101980 tx=0x7f2678009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:51.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.554+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f267c095db0 con 0x7f2688101b20 2023-08-25T04:42:51.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T04:42:51.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:51.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:51.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:51.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T04:42:51.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-08-25T04:42:51.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-08-25T04:42:51.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:51 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-25T04:42:51.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T04:42:51.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:51.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:51.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:51.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T04:42:51.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-08-25T04:42:51.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-08-25T04:42:51.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:51 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-25T04:42:51.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: osdmap e82: 12 total, 12 up, 12 in 2023-08-25T04:42:51.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:42:51.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:42:51.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:42:51.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:51.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-25T04:42:51.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-08-25T04:42:51.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-08-25T04:42:51.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:51 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-25T04:42:51.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:51.760+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f268810c190 con 0x7f2688101b20 2023-08-25T04:42:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-25T04:42:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T04:42:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: pgmap v262: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:52.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T04:42:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:active 2023-08-25T04:42:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:52.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:52 smithi145 ceph-mon[97681]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:52.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: pgmap v262: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:active 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:52.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:52 smithi162 ceph-mon[130941]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:52.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3870397131' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: osdmap e83: 12 total, 12 up, 12 in 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: pgmap v262: 97 pgs: 28 creating+peering, 65 unknown, 4 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] up:active 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:52.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:52 smithi183 ceph-mon[131414]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:53.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.330+0000 7f266ffff700 1 -- 172.21.15.145:0/3248922454 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7f267c0993d0 con 0x7f2688101b20 2023-08-25T04:42:53.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.338+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 msgr2=0x7f26700a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:53.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.338+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 0x7f26700a2530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2688101980 tx=0x7f2678009260 comp rx=0 tx=0).stop 2023-08-25T04:42:53.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.338+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 msgr2=0x7f2688198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:53.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.338+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c00f690 comp rx=0 tx=0).stop 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 shutdown_connections 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f26700a0070 0x7f26700a2530 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26881024c0 0x7f268819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2688100930 0x7f2688198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 --2- 172.21.15.145:0/3248922454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2688101b20 0x7f2688198d50 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c00f690 comp rx=0 tx=0).stop 2023-08-25T04:42:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.339+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 >> 172.21.15.145:0/3248922454 conn(0x7f26880fbe60 msgr2=0x7f26881073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.340+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 shutdown_connections 2023-08-25T04:42:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.340+0000 7f268d1ff700 1 -- 172.21.15.145:0/3248922454 wait complete. 2023-08-25T04:42:53.343 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data_ec' created 2023-08-25T04:42:53.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-25T04:42:53.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 -- 172.21.15.145:0/297277124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c106660 msgr2=0x7f5d0c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:53.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 --2- 172.21.15.145:0/297277124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c106660 0x7f5d0c108ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b0a0 tx=0x7f5d0400b3b0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 -- 172.21.15.145:0/297277124 shutdown_connections 2023-08-25T04:42:53.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 --2- 172.21.15.145:0/297277124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c106660 0x7f5d0c108ad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 --2- 172.21.15.145:0/297277124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c101400 0x7f5d0c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 --2- 172.21.15.145:0/297277124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 0x7f5d0c106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 -- 172.21.15.145:0/297277124 >> 172.21.15.145:0/297277124 conn(0x7f5d0c0fadf0 msgr2=0x7f5d0c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:53.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 -- 172.21.15.145:0/297277124 shutdown_connections 2023-08-25T04:42:53.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.563+0000 7f5d13dff700 1 -- 172.21.15.145:0/297277124 wait complete. 2023-08-25T04:42:53.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.564+0000 7f5d13dff700 1 Processor -- start 2023-08-25T04:42:53.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.564+0000 7f5d13dff700 1 -- start start 2023-08-25T04:42:53.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:53.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 0x7f5d0c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:53.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c106660 0x7f5d0c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c19e250 con 0x7f5d0c101400 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c19e390 con 0x7f5d0c106660 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d13dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c19e520 con 0x7f5d0c103d30 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58058/0 (socket says 172.21.15.145:58058) 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 -- 172.21.15.145:0/2755664415 learned_addr learned my addr 172.21.15.145:0/2755664415 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:53.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d1239c700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c106660 0x7f5d0c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d1139a700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 0x7f5d0c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 -- 172.21.15.145:0/2755664415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 msgr2=0x7f5d0c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 0x7f5d0c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 -- 172.21.15.145:0/2755664415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c106660 msgr2=0x7f5d0c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c106660 0x7f5d0c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.565+0000 7f5d11b9b700 1 -- 172.21.15.145:0/2755664415 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04009d00 con 0x7f5d0c101400 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d11b9b700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9a0 tx=0x7f5d0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:53.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08009790 con 0x7f5d0c101400 2023-08-25T04:42:53.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c19e800 con 0x7f5d0c101400 2023-08-25T04:42:53.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d0800cbd0 con 0x7f5d0c101400 2023-08-25T04:42:53.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08018680 con 0x7f5d0c101400 2023-08-25T04:42:53.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.566+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c19ed60 con 0x7f5d0c101400 2023-08-25T04:42:53.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.567+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c0fc970 con 0x7f5d0c101400 2023-08-25T04:42:53.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.569+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5d0800cd40 con 0x7f5d0c101400 2023-08-25T04:42:53.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.570+0000 7f5d02ffd700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 0x7f5cf80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:53.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.570+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f5d08014070 con 0x7f5d0c101400 2023-08-25T04:42:53.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.571+0000 7f5d1139a700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 0x7f5cf80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:53.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.571+0000 7f5d1139a700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 0x7f5cf80a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009bf0 tx=0x7f5cfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:53.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.573+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5d080954c0 con 0x7f5d0c101400 2023-08-25T04:42:53.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:53 smithi145 ceph-mon[97681]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-08-25T04:42:53.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:53 smithi145 ceph-mon[97681]: Cluster is now healthy 2023-08-25T04:42:53.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:53 smithi145 ceph-mon[97681]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T04:42:53.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:53 smithi162 ceph-mon[130941]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-08-25T04:42:53.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:53 smithi162 ceph-mon[130941]: Cluster is now healthy 2023-08-25T04:42:53.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:53 smithi162 ceph-mon[130941]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T04:42:53.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:53 smithi183 ceph-mon[131414]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-08-25T04:42:53.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:53 smithi183 ceph-mon[131414]: Cluster is now healthy 2023-08-25T04:42:53.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:53 smithi183 ceph-mon[131414]: osdmap e84: 12 total, 12 up, 12 in 2023-08-25T04:42:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:53.774+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5d0c10c0f0 con 0x7f5d0c101400 2023-08-25T04:42:54.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.338+0000 7f5d02ffd700 1 -- 172.21.15.145:0/2755664415 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d080b8020 con 0x7f5d0c101400 2023-08-25T04:42:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.355+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 msgr2=0x7f5cf80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.356+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 0x7f5cf80a2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009bf0 tx=0x7f5cfc009260 comp rx=0 tx=0).stop 2023-08-25T04:42:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.356+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 msgr2=0x7f5d0c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.356+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9a0 tx=0x7f5d0800ecb0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 shutdown_connections 2023-08-25T04:42:54.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5cf809ff50 0x7f5cf80a2410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d0c106660 0x7f5d0c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d0c101400 0x7f5d0c1966d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 --2- 172.21.15.145:0/2755664415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d0c103d30 0x7f5d0c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.359+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 >> 172.21.15.145:0/2755664415 conn(0x7f5d0c0fadf0 msgr2=0x7f5d0c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:54.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.361+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 shutdown_connections 2023-08-25T04:42:54.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.361+0000 7f5d13dff700 1 -- 172.21.15.145:0/2755664415 wait complete. 2023-08-25T04:42:54.364 INFO:teuthology.orchestra.run.smithi145.stderr:set pool 4 allow_ec_overwrites to true 2023-08-25T04:42:54.382 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-25T04:42:54.582 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T04:42:54.582 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:54 smithi145 ceph-mon[97681]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T04:42:54.582 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:54 smithi145 ceph-mon[97681]: pgmap v265: 129 pgs: 32 creating+peering, 31 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-08-25T04:42:54.583 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-25T04:42:54.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 -- 172.21.15.145:0/571872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb724100930 msgr2=0x7fb724100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:54.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 --2- 172.21.15.145:0/571872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb724100930 0x7fb724100d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7180099c0 tx=0x7fb718009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 -- 172.21.15.145:0/571872492 shutdown_connections 2023-08-25T04:42:54.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 --2- 172.21.15.145:0/571872492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7241024c0 0x7fb724108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 --2- 172.21.15.145:0/571872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb724100930 0x7fb724100d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 --2- 172.21.15.145:0/571872492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724101b20 0x7fb724101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 -- 172.21.15.145:0/571872492 >> 172.21.15.145:0/571872492 conn(0x7fb7240fbea0 msgr2=0x7fb7240fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:54.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 -- 172.21.15.145:0/571872492 shutdown_connections 2023-08-25T04:42:54.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.585+0000 7fb72abff700 1 -- 172.21.15.145:0/571872492 wait complete. 2023-08-25T04:42:54.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 Processor -- start 2023-08-25T04:42:54.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 -- start start 2023-08-25T04:42:54.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 0x7fb7241988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:54.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb724101b20 0x7fb724198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:54.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 0x7fb72419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241a0420 con 0x7fb7241024c0 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241a0560 con 0x7fb724100930 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.586+0000 7fb72abff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241a06f0 con 0x7fb724101b20 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72899b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 0x7fb7241988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72899b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 0x7fb7241988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:33076/0 (socket says 172.21.15.145:33076) 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72899b700 1 -- 172.21.15.145:0/607521635 learned_addr learned my addr 172.21.15.145:0/607521635 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 0x7fb72419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:54.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb723fff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb724101b20 0x7fb724198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 -- 172.21.15.145:0/607521635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb724101b20 msgr2=0x7fb724198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb724101b20 0x7fb724198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 -- 172.21.15.145:0/607521635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 msgr2=0x7fb7241988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 0x7fb7241988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 -- 172.21.15.145:0/607521635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180095f0 con 0x7fb7241024c0 2023-08-25T04:42:54.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb72919c700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 0x7fb72419fe50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00efc0 tx=0x7fb71c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.587+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c018070 con 0x7fb7241024c0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.588+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71c00f040 con 0x7fb7241024c0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.588+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00cd40 con 0x7fb7241024c0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.588+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241a0970 con 0x7fb7241024c0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.588+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241a0e30 con 0x7fb7241024c0 2023-08-25T04:42:54.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.589+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72410c190 con 0x7fb7241024c0 2023-08-25T04:42:54.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.591+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fb71c003a00 con 0x7fb7241024c0 2023-08-25T04:42:54.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.592+0000 7fb721ffb700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 0x7fb7140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:54.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.592+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb71c0cb8c0 con 0x7fb7241024c0 2023-08-25T04:42:54.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.593+0000 7fb72899b700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 0x7fb7140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:54.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.593+0000 7fb72899b700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 0x7fb7140a2490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb718000c00 tx=0x7fb718005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:54.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.595+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb71c095510 con 0x7fb7241024c0 2023-08-25T04:42:54.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T04:42:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:54 smithi162 ceph-mon[130941]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T04:42:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:54 smithi162 ceph-mon[130941]: pgmap v265: 129 pgs: 32 creating+peering, 31 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-08-25T04:42:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-25T04:42:54.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3248922454' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-25T04:42:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:54 smithi183 ceph-mon[131414]: osdmap e85: 12 total, 12 up, 12 in 2023-08-25T04:42:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:54 smithi183 ceph-mon[131414]: pgmap v265: 129 pgs: 32 creating+peering, 31 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-08-25T04:42:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-25T04:42:54.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:54.842+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb72410c3a0 con 0x7fb7241024c0 2023-08-25T04:42:55.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.537+0000 7fb721ffb700 1 -- 172.21.15.145:0/607521635 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb71c098b30 con 0x7fb7241024c0 2023-08-25T04:42:55.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 msgr2=0x7fb7140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:55.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 0x7fb7140a2490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb718000c00 tx=0x7fb718005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 msgr2=0x7fb72419fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:55.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 0x7fb72419fe50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00efc0 tx=0x7fb71c00c3a0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 shutdown_connections 2023-08-25T04:42:55.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb71409ffd0 0x7fb7140a2490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7241024c0 0x7fb72419fe50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb724100930 0x7fb7241988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 --2- 172.21.15.145:0/607521635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb724101b20 0x7fb724198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 >> 172.21.15.145:0/607521635 conn(0x7fb7240fbea0 msgr2=0x7fb7240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:55.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 shutdown_connections 2023-08-25T04:42:55.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.540+0000 7fb72abff700 1 -- 172.21.15.145:0/607521635 wait complete. 2023-08-25T04:42:55.544 INFO:teuthology.orchestra.run.smithi145.stderr:added data pool 4 to fsmap 2023-08-25T04:42:55.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:55.620 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:55 smithi145 ceph-mon[97681]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:55.622 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:55 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-25T04:42:55.623 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:55 smithi145 ceph-mon[97681]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T04:42:55.624 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:55 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-25T04:42:55.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:55 smithi162 ceph-mon[130941]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:55.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:55 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-25T04:42:55.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:55 smithi162 ceph-mon[130941]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T04:42:55.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:55 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-25T04:42:55.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:55 smithi183 ceph-mon[131414]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-08-25T04:42:55.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:55 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2755664415' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-25T04:42:55.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:55 smithi183 ceph-mon[131414]: osdmap e86: 12 total, 12 up, 12 in 2023-08-25T04:42:55.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:55 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-25T04:42:55.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 -- 172.21.15.145:0/3306893747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c101980 msgr2=0x7f5a4c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:55.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/3306893747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c101980 0x7f5a4c101de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a380099c0 tx=0x7f5a38009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 -- 172.21.15.145:0/3306893747 shutdown_connections 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/3306893747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c102320 0x7f5a4c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/3306893747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c100790 0x7f5a4c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/3306893747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c101980 0x7f5a4c101de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 -- 172.21.15.145:0/3306893747 >> 172.21.15.145:0/3306893747 conn(0x7f5a4c0fbce0 msgr2=0x7f5a4c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:55.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 -- 172.21.15.145:0/3306893747 shutdown_connections 2023-08-25T04:42:55.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.757+0000 7f5a51f9d700 1 -- 172.21.15.145:0/3306893747 wait complete. 2023-08-25T04:42:55.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.758+0000 7f5a51f9d700 1 Processor -- start 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 -- start start 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c101980 0x7f5a4c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 0x7f5a4c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19dea0 con 0x7f5a4c100790 2023-08-25T04:42:55.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19dfe0 con 0x7f5a4c102320 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a51f9d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c19e170 con 0x7f5a4c101980 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a50f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a50f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58092/0 (socket says 172.21.15.145:58092) 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a50f9b700 1 -- 172.21.15.145:0/1263152430 learned_addr learned my addr 172.21.15.145:0/1263152430 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.759+0000 7f5a4bfff700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c101980 0x7f5a4c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a5179c700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 0x7f5a4c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:55.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 -- 172.21.15.145:0/1263152430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c101980 msgr2=0x7f5a4c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c101980 0x7f5a4c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 -- 172.21.15.145:0/1263152430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 msgr2=0x7f5a4c19d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 0x7f5a4c19d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 -- 172.21.15.145:0/1263152430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a380095f0 con 0x7f5a4c100790 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a5179c700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 0x7f5a4c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:55.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a50f9b700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a40007ee0 tx=0x7f5a4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:55.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000ddd0 con 0x7f5a4c100790 2023-08-25T04:42:55.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c19e450 con 0x7f5a4c100790 2023-08-25T04:42:55.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a4000f040 con 0x7f5a4c100790 2023-08-25T04:42:55.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40014690 con 0x7f5a4c100790 2023-08-25T04:42:55.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.760+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c19e9b0 con 0x7f5a4c100790 2023-08-25T04:42:55.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.761+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c107550 con 0x7f5a4c100790 2023-08-25T04:42:55.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.763+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5a400159c0 con 0x7f5a4c100790 2023-08-25T04:42:55.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.764+0000 7f5a49ffb700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 0x7f5a3c0a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:55.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.765+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5a400cafa0 con 0x7f5a4c100790 2023-08-25T04:42:55.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.765+0000 7f5a4bfff700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 0x7f5a3c0a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:55.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.765+0000 7f5a4bfff700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 0x7f5a3c0a2500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b580 tx=0x7f5a38005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:55.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:55.768+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5a40094ae0 con 0x7f5a4c100790 2023-08-25T04:42:56.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.010+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a4c10c000 con 0x7f5a4c100790 2023-08-25T04:42:56.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.011+0000 7f5a49ffb700 1 -- 172.21.15.145:0/1263152430 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11241 (secure 0 0 0) 0x7f5a400abe60 con 0x7f5a4c100790 2023-08-25T04:42:56.013 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:56.013 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:42:55.534078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-25T04:42:56.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 msgr2=0x7f5a3c0a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 0x7f5a3c0a2500 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b580 tx=0x7f5a38005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 msgr2=0x7f5a4c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a40007ee0 tx=0x7f5a4000d3b0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 shutdown_connections 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a3c0a0040 0x7f5a3c0a2500 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a4c102320 0x7f5a4c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a4c100790 0x7f5a4c196320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 --2- 172.21.15.145:0/1263152430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a4c101980 0x7f5a4c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 >> 172.21.15.145:0/1263152430 conn(0x7f5a4c0fbce0 msgr2=0x7f5a4c0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:56.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 shutdown_connections 2023-08-25T04:42:56.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.014+0000 7f5a51f9d700 1 -- 172.21.15.145:0/1263152430 wait complete. 2023-08-25T04:42:56.017 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2023-08-25T04:42:56.033 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:42:56.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 -- 172.21.15.145:0/132156167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c101b20 msgr2=0x7fc23c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 --2- 172.21.15.145:0/132156167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c101b20 0x7fc23c101f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc2280099c0 tx=0x7fc228009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 -- 172.21.15.145:0/132156167 shutdown_connections 2023-08-25T04:42:56.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 --2- 172.21.15.145:0/132156167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c1024c0 0x7fc23c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 --2- 172.21.15.145:0/132156167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c100930 0x7fc23c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 --2- 172.21.15.145:0/132156167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c101b20 0x7fc23c101f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 -- 172.21.15.145:0/132156167 >> 172.21.15.145:0/132156167 conn(0x7fc23c0fbe60 msgr2=0x7fc23c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:56.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 -- 172.21.15.145:0/132156167 shutdown_connections 2023-08-25T04:42:56.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.231+0000 7fc242bff700 1 -- 172.21.15.145:0/132156167 wait complete. 2023-08-25T04:42:56.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 Processor -- start 2023-08-25T04:42:56.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 -- start start 2023-08-25T04:42:56.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 0x7fc23c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c1024c0 0x7fc23c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0739f0 con 0x7fc23c1024c0 2023-08-25T04:42:56.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c073b60 con 0x7fc23c101b20 2023-08-25T04:42:56.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc242bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c1a0630 con 0x7fc23c100930 2023-08-25T04:42:56.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.232+0000 7fc24119c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c1024c0 0x7fc23c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42860/0 (socket says 172.21.15.145:42860) 2023-08-25T04:42:56.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 -- 172.21.15.145:0/2868469279 learned_addr learned my addr 172.21.15.145:0/2868469279 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:56.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc23bfff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 0x7fc23c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 -- 172.21.15.145:0/2868469279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 msgr2=0x7fc23c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 0x7fc23c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 -- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c1024c0 msgr2=0x7fc23c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c1024c0 0x7fc23c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 -- 172.21.15.145:0/2868469279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280095f0 con 0x7fc23c100930 2023-08-25T04:42:56.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc23bfff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 0x7fc23c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:56.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc24099b700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e970 tx=0x7fc23000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:56.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230009790 con 0x7fc23c100930 2023-08-25T04:42:56.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c1a0910 con 0x7fc23c100930 2023-08-25T04:42:56.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc23000eec0 con 0x7fc23c100930 2023-08-25T04:42:56.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230003cf0 con 0x7fc23c100930 2023-08-25T04:42:56.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.233+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c1a0e70 con 0x7fc23c100930 2023-08-25T04:42:56.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.234+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c107ea0 con 0x7fc23c100930 2023-08-25T04:42:56.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.237+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc23000ccd0 con 0x7fc23c100930 2023-08-25T04:42:56.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.238+0000 7fc239ffb700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 0x7fc22c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.238+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc230014070 con 0x7fc23c100930 2023-08-25T04:42:56.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.238+0000 7fc23bfff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 0x7fc22c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.239+0000 7fc23bfff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 0x7fc22c0a2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc22800b580 tx=0x7fc22801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:56.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.240+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc230095360 con 0x7fc23c100930 2023-08-25T04:42:56.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.443+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc23c1a0aa0 con 0x7fc23c100930 2023-08-25T04:42:56.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.445+0000 7fc239ffb700 1 -- 172.21.15.145:0/2868469279 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7fc23c1a0aa0 con 0x7fc23c100930 2023-08-25T04:42:56.446 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:56.446 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:56.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 msgr2=0x7fc22c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 0x7fc22c0a2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc22800b580 tx=0x7fc22801a040 comp rx=0 tx=0).stop 2023-08-25T04:42:56.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 msgr2=0x7fc23c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e970 tx=0x7fc23000ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:56.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 shutdown_connections 2023-08-25T04:42:56.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc22c09ffa0 0x7fc22c0a2460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc23c1024c0 0x7fc23c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc23c100930 0x7fc23c0746d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 --2- 172.21.15.145:0/2868469279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc23c101b20 0x7fc23c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 >> 172.21.15.145:0/2868469279 conn(0x7fc23c0fbe60 msgr2=0x7fc23c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:56.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 shutdown_connections 2023-08-25T04:42:56.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.447+0000 7fc242bff700 1 -- 172.21.15.145:0/2868469279 wait complete. 2023-08-25T04:42:56.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:42:56.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:56 smithi162 ceph-mon[130941]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T04:42:56.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-25T04:42:56.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:56 smithi162 ceph-mon[130941]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:56.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:56 smithi162 ceph-mon[130941]: pgmap v268: 129 pgs: 19 creating+peering, 10 unknown, 100 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-08-25T04:42:56.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1263152430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:56.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 -- 172.21.15.145:0/2174948328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc1024c0 msgr2=0x7f7ebc108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/2174948328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc108d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b0d0 tx=0x7f7eb400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 -- 172.21.15.145:0/2174948328 shutdown_connections 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/2174948328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc108d70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/2174948328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc100930 0x7f7ebc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/2174948328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 -- 172.21.15.145:0/2174948328 >> 172.21.15.145:0/2174948328 conn(0x7f7ebc0fbe60 msgr2=0x7f7ebc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 -- 172.21.15.145:0/2174948328 shutdown_connections 2023-08-25T04:42:56.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.667+0000 7f7ec09ff700 1 -- 172.21.15.145:0/2174948328 wait complete. 2023-08-25T04:42:56.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.668+0000 7f7ec09ff700 1 Processor -- start 2023-08-25T04:42:56.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 -- start start 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc1a0460 con 0x7f7ebc100930 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc1a05a0 con 0x7f7ebc101b20 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ec09ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc1a0730 con 0x7f7ebc1024c0 2023-08-25T04:42:56.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7eba59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7eba59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58126/0 (socket says 172.21.15.145:58126) 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7eba59c700 1 -- 172.21.15.145:0/1866534469 learned_addr learned my addr 172.21.15.145:0/1866534469 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7eb9d9b700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.669+0000 7f7ebad9d700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 -- 172.21.15.145:0/1866534469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc1024c0 msgr2=0x7f7ebc19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc19fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 -- 172.21.15.145:0/1866534469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 msgr2=0x7f7ebc198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 -- 172.21.15.145:0/1866534469 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4009d00 con 0x7f7ebc100930 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eb9d9b700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eba59c700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e970 tx=0x7f7eac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac009790 con 0x7f7ebc100930 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eac00c7e0 con 0x7f7ebc100930 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc1a0a10 con 0x7f7ebc100930 2023-08-25T04:42:56.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac003c70 con 0x7f7ebc100930 2023-08-25T04:42:56.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.670+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc1a0f70 con 0x7f7ebc100930 2023-08-25T04:42:56.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:56 smithi183 ceph-mon[131414]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T04:42:56.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-25T04:42:56.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:56 smithi183 ceph-mon[131414]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:56.676 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:56 smithi183 ceph-mon[131414]: pgmap v268: 129 pgs: 19 creating+peering, 10 unknown, 100 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-08-25T04:42:56.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1263152430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.671+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc10c1a0 con 0x7f7ebc100930 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.673+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f7eac00cc50 con 0x7f7ebc100930 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.674+0000 7f7eab7fe700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 0x7f7ea40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.674+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7eac014070 con 0x7f7ebc100930 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.675+0000 7f7eb9d9b700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 0x7f7ea40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:56.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.675+0000 7f7eb9d9b700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 0x7f7ea40a24e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011490 tx=0x7f7eb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:56.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.678+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7eac095100 con 0x7f7ebc100930 2023-08-25T04:42:56.828 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: 2023-08-25T04:42:56.546+0000 7f558a4a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T04:42:56.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-mon[97681]: osdmap e87: 12 total, 12 up, 12 in 2023-08-25T04:42:56.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/607521635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-25T04:42:56.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-mon[97681]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:42:56.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-mon[97681]: pgmap v268: 129 pgs: 19 creating+peering, 10 unknown, 100 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-08-25T04:42:56.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1263152430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:56.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.883+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ebc1a11c0 con 0x7f7ebc100930 2023-08-25T04:42:56.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.885+0000 7f7eab7fe700 1 -- 172.21.15.145:0/1866534469 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f7eac003dd0 con 0x7f7ebc100930 2023-08-25T04:42:56.886 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:56.886 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:56.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.887+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 msgr2=0x7f7ea40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.887+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 0x7f7ea40a24e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011490 tx=0x7f7eb0009260 comp rx=0 tx=0).stop 2023-08-25T04:42:56.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 msgr2=0x7f7ebc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:56.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e970 tx=0x7f7eac00ec80 comp rx=0 tx=0).stop 2023-08-25T04:42:56.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 shutdown_connections 2023-08-25T04:42:56.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f7ea40a0020 0x7f7ea40a24e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ebc1024c0 0x7f7ebc19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ebc100930 0x7f7ebc1988e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 --2- 172.21.15.145:0/1866534469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ebc101b20 0x7f7ebc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:56.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 >> 172.21.15.145:0/1866534469 conn(0x7f7ebc0fbe60 msgr2=0x7f7ebc107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:56.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 shutdown_connections 2023-08-25T04:42:56.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:56.888+0000 7f7ec09ff700 1 -- 172.21.15.145:0/1866534469 wait complete. 2023-08-25T04:42:56.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:42:57.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 -- 172.21.15.145:0/3087767492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 msgr2=0x7f0840101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 --2- 172.21.15.145:0/3087767492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840101f90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08340099c0 tx=0x7f0834009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 -- 172.21.15.145:0/3087767492 shutdown_connections 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 --2- 172.21.15.145:0/3087767492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08401024d0 0x7f0840108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 --2- 172.21.15.145:0/3087767492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0840100940 0x7f0840100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 --2- 172.21.15.145:0/3087767492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840101f90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 -- 172.21.15.145:0/3087767492 >> 172.21.15.145:0/3087767492 conn(0x7f08400fbe90 msgr2=0x7f08400fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 -- 172.21.15.145:0/3087767492 shutdown_connections 2023-08-25T04:42:57.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.131+0000 7f0845bff700 1 -- 172.21.15.145:0/3087767492 wait complete. 2023-08-25T04:42:57.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.132+0000 7f0845bff700 1 Processor -- start 2023-08-25T04:42:57.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.132+0000 7f0845bff700 1 -- start start 2023-08-25T04:42:57.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0840100940 0x7f08401965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 0x7f084019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019e140 con 0x7f0840101b30 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019e280 con 0x7f08401024d0 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f0845bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019e410 con 0x7f0840100940 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58156/0 (socket says 172.21.15.145:58156) 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 -- 172.21.15.145:0/2886972089 learned_addr learned my addr 172.21.15.145:0/2886972089 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083f7fe700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0840100940 0x7f08401965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083ffff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 0x7f084019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 -- 172.21.15.145:0/2886972089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0840100940 msgr2=0x7f08401965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0840100940 0x7f08401965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 -- 172.21.15.145:0/2886972089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 msgr2=0x7f084019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 0x7f084019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083effd700 1 -- 172.21.15.145:0/2886972089 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08340095f0 con 0x7f0840101b30 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.133+0000 7f083ffff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 0x7f084019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:57.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f083effd700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0834009fd0 tx=0x7f083400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:57.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083401d070 con 0x7f0840101b30 2023-08-25T04:42:57.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0834005510 con 0x7f0840101b30 2023-08-25T04:42:57.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084019e690 con 0x7f0840101b30 2023-08-25T04:42:57.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834004ae0 con 0x7f0840101b30 2023-08-25T04:42:57.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.134+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084019eab0 con 0x7f0840101b30 2023-08-25T04:42:57.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.135+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0840107e90 con 0x7f0840101b30 2023-08-25T04:42:57.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.137+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f0834004cc0 con 0x7f0840101b30 2023-08-25T04:42:57.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.138+0000 7f083cff9700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 0x7f08280a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.138+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f08340cc290 con 0x7f0840101b30 2023-08-25T04:42:57.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.138+0000 7f083f7fe700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 0x7f08280a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.139+0000 7f083f7fe700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 0x7f08280a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0840101990 tx=0x7f0830009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:57.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.142+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0834095ec0 con 0x7f0840101b30 2023-08-25T04:42:57.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.391+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f084010c1a0 con 0x7f0840101b30 2023-08-25T04:42:57.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.392+0000 7f083cff9700 1 -- 172.21.15.145:0/2886972089 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11241 (secure 0 0 0) 0x7f08340994e0 con 0x7f0840101b30 2023-08-25T04:42:57.397 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:57.397 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:42:55.534078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-25T04:42:57.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 msgr2=0x7f08280a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 0x7f08280a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0840101990 tx=0x7f0830009260 comp rx=0 tx=0).stop 2023-08-25T04:42:57.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 msgr2=0x7f0840196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0834009fd0 tx=0x7f083400f690 comp rx=0 tx=0).stop 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 shutdown_connections 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f082809ffa0 0x7f08280a2460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08401024d0 0x7f084019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0840100940 0x7f08401965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 --2- 172.21.15.145:0/2886972089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0840101b30 0x7f0840196b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.394+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 >> 172.21.15.145:0/2886972089 conn(0x7f08400fbe90 msgr2=0x7f0840104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:57.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.395+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 shutdown_connections 2023-08-25T04:42:57.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.395+0000 7f0845bff700 1 -- 172.21.15.145:0/2886972089 wait complete. 2023-08-25T04:42:57.400 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2023-08-25T04:42:57.414 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:42:57.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 -- 172.21.15.145:0/256992806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0268100930 msgr2=0x7f0268100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 --2- 172.21.15.145:0/256992806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0268100930 0x7f0268100d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f025c0099c0 tx=0x7f025c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 -- 172.21.15.145:0/256992806 shutdown_connections 2023-08-25T04:42:57.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 --2- 172.21.15.145:0/256992806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02681024c0 0x7f0268108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 --2- 172.21.15.145:0/256992806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0268100930 0x7f0268100d50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 --2- 172.21.15.145:0/256992806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 -- 172.21.15.145:0/256992806 >> 172.21.15.145:0/256992806 conn(0x7f02680fbe60 msgr2=0x7f02680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:57.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 -- 172.21.15.145:0/256992806 shutdown_connections 2023-08-25T04:42:57.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.612+0000 7f026f1ff700 1 -- 172.21.15.145:0/256992806 wait complete. 2023-08-25T04:42:57.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 Processor -- start 2023-08-25T04:42:57.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 -- start start 2023-08-25T04:42:57.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0268100930 0x7f0268198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02681024c0 0x7f026819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681a03f0 con 0x7f02681024c0 2023-08-25T04:42:57.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681a0530 con 0x7f0268100930 2023-08-25T04:42:57.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026f1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681a06c0 con 0x7f0268101b20 2023-08-25T04:42:57.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f0267fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f026d79c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02681024c0 0x7f026819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f0267fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42912/0 (socket says 172.21.15.145:42912) 2023-08-25T04:42:57.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.613+0000 7f0267fff700 1 -- 172.21.15.145:0/3581524108 learned_addr learned my addr 172.21.15.145:0/3581524108 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:57.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f026cf9b700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0268100930 0x7f0268198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 -- 172.21.15.145:0/3581524108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0268100930 msgr2=0x7f0268198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0268100930 0x7f0268198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 -- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02681024c0 msgr2=0x7f026819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02681024c0 0x7f026819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 -- 172.21.15.145:0/3581524108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0095f0 con 0x7f0268101b20 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0267fff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f025400e9a0 tx=0x7f025400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254009790 con 0x7f0268101b20 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0254010430 con 0x7f0268101b20 2023-08-25T04:42:57.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02681a09a0 con 0x7f0268101b20 2023-08-25T04:42:57.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.614+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254018680 con 0x7f0268101b20 2023-08-25T04:42:57.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.615+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02681a0eb0 con 0x7f0268101b20 2023-08-25T04:42:57.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.616+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268192f80 con 0x7f0268101b20 2023-08-25T04:42:57.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.618+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f02540105a0 con 0x7f0268101b20 2023-08-25T04:42:57.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.619+0000 7f0265ffb700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 0x7f02580a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:57.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.619+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0254014070 con 0x7f0268101b20 2023-08-25T04:42:57.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.619+0000 7f026cf9b700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 0x7f02580a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:57.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.620+0000 7f026cf9b700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 0x7f02580a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f025c009990 tx=0x7f025c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:57.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.621+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f02540950d0 con 0x7f0268101b20 2023-08-25T04:42:57.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:57 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2868469279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:57 smithi162 ceph-mon[130941]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T04:42:57.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:57 smithi162 ceph-mon[130941]: Cluster is now healthy 2023-08-25T04:42:57.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:57 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1866534469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:57 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2868469279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:57 smithi183 ceph-mon[131414]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T04:42:57.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:57 smithi183 ceph-mon[131414]: Cluster is now healthy 2023-08-25T04:42:57.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:57 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1866534469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:57 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2868469279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:57 smithi145 ceph-mon[97681]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-08-25T04:42:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:57 smithi145 ceph-mon[97681]: Cluster is now healthy 2023-08-25T04:42:57.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:57 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1866534469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.827+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f026810c3e0 con 0x7f0268101b20 2023-08-25T04:42:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.829+0000 7f0265ffb700 1 -- 172.21.15.145:0/3581524108 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f02540986f0 con 0x7f0268101b20 2023-08-25T04:42:57.831 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:42:57.831 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 msgr2=0x7f02580a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 0x7f02580a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f025c009990 tx=0x7f025c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 msgr2=0x7f0268198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f025400e9a0 tx=0x7f025400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 shutdown_connections 2023-08-25T04:42:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f025809ffa0 0x7f02580a2460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02681024c0 0x7f026819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0268100930 0x7f0268198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 --2- 172.21.15.145:0/3581524108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0268101b20 0x7f0268198db0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 >> 172.21.15.145:0/3581524108 conn(0x7f02680fbe60 msgr2=0x7f0268107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 shutdown_connections 2023-08-25T04:42:57.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:57.832+0000 7f026f1ff700 1 -- 172.21.15.145:0/3581524108 wait complete. 2023-08-25T04:42:57.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T04:42:49.862390+0000', 'modified': '2023-08-25T04:42:55.534078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24562': {'gid': 24562, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.145:6841/3986940142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6840', 'nonce': 3986940142}, {'type': 'v1', 'addr': '172.21.15.145:6841', 'nonce': 3986940142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-25T04:42:57.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T04:42:57.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T04:42:58.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 -- 172.21.15.145:0/379332358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 msgr2=0x7f5b3c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 --2- 172.21.15.145:0/379332358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c101f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b300099c0 tx=0x7f5b30009cd0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 -- 172.21.15.145:0/379332358 shutdown_connections 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 --2- 172.21.15.145:0/379332358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c1024c0 0x7f5b3c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 --2- 172.21.15.145:0/379332358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c100930 0x7f5b3c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 --2- 172.21.15.145:0/379332358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c101f80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 -- 172.21.15.145:0/379332358 >> 172.21.15.145:0/379332358 conn(0x7f5b3c0fbea0 msgr2=0x7f5b3c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 -- 172.21.15.145:0/379332358 shutdown_connections 2023-08-25T04:42:58.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.051+0000 7f5b417ff700 1 -- 172.21.15.145:0/379332358 wait complete. 2023-08-25T04:42:58.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.052+0000 7f5b417ff700 1 Processor -- start 2023-08-25T04:42:58.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.052+0000 7f5b417ff700 1 -- start start 2023-08-25T04:42:58.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.052+0000 7f5b417ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 0x7f5b3c1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.052+0000 7f5b417ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.052+0000 7f5b417ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c1024c0 0x7f5b3c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b417ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c19e240 con 0x7f5b3c101b20 2023-08-25T04:42:58.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b417ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c19e380 con 0x7f5b3c100930 2023-08-25T04:42:58.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b417ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c19e510 con 0x7f5b3c1024c0 2023-08-25T04:42:58.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 0x7f5b3c1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58188/0 (socket says 172.21.15.145:58188) 2023-08-25T04:42:58.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/63433383 learned_addr learned my addr 172.21.15.145:0/63433383 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:58.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c1024c0 0x7f5b3c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/63433383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c1024c0 msgr2=0x7f5b3c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c1024c0 0x7f5b3c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/63433383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 msgr2=0x7f5b3c1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 0x7f5b3c1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/63433383 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300095f0 con 0x7f5b3c101b20 2023-08-25T04:42:58.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3affd700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 0x7f5b3c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:58.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.053+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009990 tx=0x7f5b3000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.054+0000 7f5b23fff700 1 -- 172.21.15.145:0/63433383 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3001d070 con 0x7f5b3c101b20 2023-08-25T04:42:58.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.054+0000 7f5b23fff700 1 -- 172.21.15.145:0/63433383 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b300054f0 con 0x7f5b3c101b20 2023-08-25T04:42:58.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.054+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c19e790 con 0x7f5b3c101b20 2023-08-25T04:42:58.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.054+0000 7f5b23fff700 1 -- 172.21.15.145:0/63433383 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30004b00 con 0x7f5b3c101b20 2023-08-25T04:42:58.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.054+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c19ebb0 con 0x7f5b3c101b20 2023-08-25T04:42:58.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.057+0000 7f5b23fff700 1 -- 172.21.15.145:0/63433383 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5b3000fb80 con 0x7f5b3c101b20 2023-08-25T04:42:58.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.058+0000 7f5b23fff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 0x7f5b240a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.058+0000 7f5b551bcb80 1 Processor -- start 2023-08-25T04:42:58.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.058+0000 7f5b23fff700 1 -- 172.21.15.145:0/63433383 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b300cc220 con 0x7f5b3c101b20 2023-08-25T04:42:58.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.058+0000 7f5b3affd700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 0x7f5b240a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.058+0000 7f5b551bcb80 1 -- start start 2023-08-25T04:42:58.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 0x56554d829da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82a2e0 0x56554d82a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d791760 con 0x56554d82ad00 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d782730 con 0x56554d82a2e0 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d780050 con 0x56554d829980 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3affd700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 0x7f5b240a2410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c101980 tx=0x7f5b2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58194/0 (socket says 172.21.15.145:58194) 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 -- 172.21.15.145:0/473995758 learned_addr learned my addr 172.21.15.145:0/473995758 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82a2e0 0x56554d82a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3affd700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 0x56554d829da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 -- 172.21.15.145:0/473995758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 msgr2=0x56554d829da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 0x56554d829da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 -- 172.21.15.145:0/473995758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82a2e0 msgr2=0x56554d82a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82a2e0 0x56554d82a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3b7fe700 1 -- 172.21.15.145:0/473995758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554d5fb5e0 con 0x56554d82ad00 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.059+0000 7f5b3affd700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 0x56554d829da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009440 tx=0x7f5b34009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd3265517a83a01 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b217fa700 1 -- 172.21.15.145:0/473995758 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400e070 con 0x56554d82ad00 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b217fa700 1 -- 172.21.15.145:0/473995758 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3400f040 con 0x56554d82ad00 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b217fa700 1 -- 172.21.15.145:0/473995758 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34015ac0 con 0x56554d82ad00 2023-08-25T04:42:58.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b551bcb80 1 -- 172.21.15.145:0/473995758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 msgr2=0x56554d82b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.060+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009440 tx=0x7f5b34009920 comp rx=0 tx=0).stop 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 -- 172.21.15.145:0/473995758 shutdown_connections 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82b180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d829980 0x56554d829da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/473995758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82a2e0 0x56554d82a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 -- 172.21.15.145:0/473995758 >> 172.21.15.145:0/473995758 conn(0x56554d79e090 msgr2=0x56554d824250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 -- 172.21.15.145:0/473995758 shutdown_connections 2023-08-25T04:42:58.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 -- 172.21.15.145:0/473995758 wait complete. 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 -- start start 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.061+0000 7f5b551bcb80 1 Processor -- start 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d8338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d83a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d77b940 con 0x56554d829980 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d719b60 con 0x56554d82ad00 2023-08-25T04:42:58.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d719860 con 0x56554d82a2e0 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b551bcb80 10 client.0 _ll_register_callbacks cb 0x56554d663a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58196/0 (socket says 172.21.15.145:58196) 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3affd700 1 -- 172.21.15.145:0/329492172 learned_addr learned my addr 172.21.15.145:0/329492172 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d83a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d8338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3affd700 1 -- 172.21.15.145:0/329492172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 msgr2=0x56554d8338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.062+0000 7f5b3affd700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d8338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b3affd700 1 -- 172.21.15.145:0/329492172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 msgr2=0x56554d83a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b3affd700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d83a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b3affd700 1 -- 172.21.15.145:0/329492172 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b340090f0 con 0x56554d829980 2023-08-25T04:42:58.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d83a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b3affd700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c003a50 tx=0x7f5b2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea5d09f75b1b6c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c01ea50 con 0x56554d829980 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b2c003e10 con 0x56554d829980 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56554d7f9170 con 0x56554d829980 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.063+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2c01ebb0 con 0x56554d829980 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 10 client.14658 fetch_fsmap learned FSMap version 17 2023-08-25T04:42:58.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56554d5a7d90 con 0x56554d829980 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56554d7f9170 con 0x56554d829980 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5b2c00e3b0 con 0x56554d829980 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 10 client.14658 fetch_fsmap finished waiting for FSMap version 17 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 10 client.14658 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 20 client.14658 populate_metadata read hostname 'smithi145' 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d83f010 0x56554d83daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 4 client.14658 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56554d721b50 con 0x56554d83f010 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.064+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d83f010 0x56554d83daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.065+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d83f010 0x56554d83daa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00ff2dacde5aa37 server_cookie=3e70e6a8d2abeb33 in_seq=0 out_seq=0 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.065+0000 7f5b1eff5700 10 client.14658 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.065+0000 7f5b1eff5700 1 -- 172.21.15.145:0/329492172 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56554d721b50 con 0x56554d83f010 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.065+0000 7f5b1eff5700 10 client.14658 handle_command_reply: tid=0 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.065+0000 7f5b551bcb80 1 client.14658 shutdown 2023-08-25T04:42:58.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 msgr2=0x56554d833370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c003a50 tx=0x7f5b2c0047c0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 shutdown_connections 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d83a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d833370 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d83f010 0x56554d83daa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/329492172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d8338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b1eff5700 0 client.14658 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:42:58.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 >> 172.21.15.145:0/329492172 conn(0x56554d79e090 msgr2=0x56554d81eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 shutdown_connections 2023-08-25T04:42:58.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 1 -- 172.21.15.145:0/329492172 wait complete. 2023-08-25T04:42:58.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.066+0000 7f5b551bcb80 20 client.14658 trim_cache size 0 max 16384 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.083+0000 7f5b551bcb80 1 Processor -- start 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 -- start start 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d829980 0x56554d831160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d5fd8d0 con 0x56554d82ad00 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d737f00 con 0x56554d829980 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d715e00 con 0x56554d82a2e0 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42968/0 (socket says 172.21.15.145:42968) 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58206/0 (socket says 172.21.15.145:58206) 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/2065733498 learned_addr learned my addr 172.21.15.145:0/2065733498 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:58.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.084+0000 7f5b3affd700 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d829980 0x56554d831160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/2065733498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d829980 msgr2=0x56554d831160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d829980 0x56554d831160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/2065733498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 msgr2=0x56554d82dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 -- 172.21.15.145:0/2065733498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554d590c10 con 0x56554d82a2e0 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:42:58.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b30000c00 tx=0x7f5b300c9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefbde2fb783ed08 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b21ffb700 1 -- 172.21.15.145:0/2065733498 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b300af070 con 0x56554d82a2e0 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b21ffb700 1 -- 172.21.15.145:0/2065733498 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b30099ab0 con 0x56554d82a2e0 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.085+0000 7f5b21ffb700 1 -- 172.21.15.145:0/2065733498 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30098fa0 con 0x56554d82a2e0 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2065733498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 msgr2=0x56554d83a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b30000c00 tx=0x7f5b300c9f90 comp rx=0 tx=0).stop 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2065733498 shutdown_connections 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d82ad00 0x56554d82dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d829980 0x56554d831160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2065733498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a8b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2065733498 >> 172.21.15.145:0/2065733498 conn(0x56554d79e090 msgr2=0x56554d8241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2065733498 shutdown_connections 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2065733498 wait complete. 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.086+0000 7f5b551bcb80 1 -- start start 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 Processor -- start 2023-08-25T04:42:58.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d836f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d837670 con 0x56554d829980 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d8377b0 con 0x56554d82ad00 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554d837970 con 0x56554d82a2e0 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b551bcb80 10 client.0 _ll_register_callbacks cb 0x7f5b2c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b3affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b3affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58208/0 (socket says 172.21.15.145:58208) 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b3affd700 1 -- 172.21.15.145:0/2225063160 learned_addr learned my addr 172.21.15.145:0/2225063160 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.087+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d836f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 -- 172.21.15.145:0/2225063160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 msgr2=0x56554d83a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 -- 172.21.15.145:0/2225063160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 msgr2=0x56554d836f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:42:58.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d836f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 -- 172.21.15.145:0/2225063160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554d5fb5e0 con 0x56554d829980 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3b7fe700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d836f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b3affd700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e740 tx=0x7f5b2c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2815e1bac8ffa54 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00eb30 con 0x56554d829980 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56554d81c860 con 0x56554d829980 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b2c004830 con 0x56554d829980 2023-08-25T04:42:58.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2c0049f0 con 0x56554d829980 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b551bcb80 10 client.14664 fetch_fsmap learned FSMap version 17 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.088+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f5b3c19ebb0 con 0x56554d829980 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56554d81c860 con 0x56554d829980 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5b2c01e6a0 con 0x56554d829980 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 10 client.14664 fetch_fsmap finished waiting for FSMap version 17 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 10 client.14664 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 20 client.14664 populate_metadata read hostname 'smithi145' 2023-08-25T04:42:58.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d897a30 0x56554d893eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:42:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 4 client.14664 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-08-25T04:42:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56554d774920 con 0x56554d897a30 2023-08-25T04:42:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d897a30 0x56554d893eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:42:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b1eff5700 10 client.14664 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:42:58.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.089+0000 7f5b3a7fc700 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d897a30 0x56554d893eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508b7c3428e1dc3f server_cookie=5b6204d212ce80af in_seq=0 out_seq=0 2023-08-25T04:42:58.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b1eff5700 1 -- 172.21.15.145:0/2225063160 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56554d774920 con 0x56554d897a30 2023-08-25T04:42:58.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b1eff5700 10 client.14664 handle_command_reply: tid=0 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 client.14664 shutdown 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 msgr2=0x56554d83a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e740 tx=0x7f5b2c00ed60 comp rx=0 tx=0).stop 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 shutdown_connections 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56554d82ad00 0x56554d836f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56554d829980 0x56554d83a400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56554d897a30 0x56554d893eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 --2- 172.21.15.145:0/2225063160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56554d82a2e0 0x56554d83a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b1eff5700 0 client.14664 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 >> 172.21.15.145:0/2225063160 conn(0x56554d79e090 msgr2=0x56554d83b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 shutdown_connections 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 1 -- 172.21.15.145:0/2225063160 wait complete. 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.090+0000 7f5b551bcb80 20 client.14664 trim_cache size 0 max 16384 2023-08-25T04:42:58.102 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24562, 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:42:58.103 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 7.2713150960000004, 2023-08-25T04:42:58.104 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 17, 2023-08-25T04:42:58.104 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 87, 2023-08-25T04:42:58.104 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:42:58.104 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 17.516992831 2023-08-25T04:42:58.104 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:42:58.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 msgr2=0x7f5b240a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 0x7f5b240a2410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c101980 tx=0x7f5b2c009260 comp rx=0 tx=0).stop 2023-08-25T04:42:58.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 msgr2=0x7f5b3c196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:42:58.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009990 tx=0x7f5b3000f690 comp rx=0 tx=0).stop 2023-08-25T04:42:58.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 shutdown_connections 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b2409ff50 0x7f5b240a2410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c1024c0 0x7f5b3c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c100930 0x7f5b3c1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 --2- 172.21.15.145:0/63433383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b3c101b20 0x7f5b3c196c00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 >> 172.21.15.145:0/63433383 conn(0x7f5b3c0fbea0 msgr2=0x7f5b3c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 shutdown_connections 2023-08-25T04:42:58.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:42:58.091+0000 7f5b417ff700 1 -- 172.21.15.145:0/63433383 wait complete. 2023-08-25T04:42:58.110 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T04:42:58.111 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:42:58.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph fs ls 2023-08-25T04:42:58.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:58 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2886972089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:58 smithi145 ceph-mon[97681]: pgmap v269: 129 pgs: 19 creating+peering, 8 unknown, 102 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2023-08-25T04:42:58.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:42:58 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3581524108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:58 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2886972089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:58 smithi162 ceph-mon[130941]: pgmap v269: 129 pgs: 19 creating+peering, 8 unknown, 102 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2023-08-25T04:42:58.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:42:58 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3581524108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:58 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2886972089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:58 smithi183 ceph-mon[131414]: pgmap v269: 129 pgs: 19 creating+peering, 8 unknown, 102 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2023-08-25T04:42:58.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:42:58 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3581524108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:42:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 -- 172.21.15.145:0/657881898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8101300 msgr2=0x7f2af810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 --2- 172.21.15.145:0/657881898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8101300 0x7f2af810d810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2af000b0a0 tx=0x7f2af000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 -- 172.21.15.145:0/657881898 shutdown_connections 2023-08-25T04:43:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 --2- 172.21.15.145:0/657881898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8101300 0x7f2af810d810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 --2- 172.21.15.145:0/657881898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 0x7f2af8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 --2- 172.21.15.145:0/657881898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8100940 0x7f2af8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.346+0000 7f2afcb21700 1 -- 172.21.15.145:0/657881898 >> 172.21.15.145:0/657881898 conn(0x7f2af80fc410 msgr2=0x7f2af80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.347+0000 7f2afcb21700 1 -- 172.21.15.145:0/657881898 shutdown_connections 2023-08-25T04:43:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.347+0000 7f2afcb21700 1 -- 172.21.15.145:0/657881898 wait complete. 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.348+0000 7f2afcb21700 1 Processor -- start 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.348+0000 7f2afcb21700 1 -- start start 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 0x7f2af8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 0x7f2af80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81a4ff0 con 0x7f2af8100940 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81a5160 con 0x7f2af8101300 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2afcb21700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81a5320 con 0x7f2af8106940 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58220/0 (socket says 172.21.15.145:58220) 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 -- 172.21.15.145:0/3659574318 learned_addr learned my addr 172.21.15.145:0/3659574318 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2af659c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 0x7f2af8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:00.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2af6d9d700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 0x7f2af80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 -- 172.21.15.145:0/3659574318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 msgr2=0x7f2af80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 0x7f2af80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 -- 172.21.15.145:0/3659574318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 msgr2=0x7f2af8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 0x7f2af8073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2aee59c700 1 -- 172.21.15.145:0/3659574318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009d00 con 0x7f2af8100940 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.349+0000 7f2af659c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 0x7f2af8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:00.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2aee59c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800e9a0 tx=0x7f2ae800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8009790 con 0x7f2af8100940 2023-08-25T04:43:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af81a5600 con 0x7f2af8100940 2023-08-25T04:43:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae8010430 con 0x7f2af8100940 2023-08-25T04:43:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8018680 con 0x7f2af8100940 2023-08-25T04:43:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.350+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af81a5b50 con 0x7f2af8100940 2023-08-25T04:43:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.351+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af804e620 con 0x7f2af8100940 2023-08-25T04:43:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.353+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f2ae80105a0 con 0x7f2af8100940 2023-08-25T04:43:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.354+0000 7f2aeffff700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 0x7f2ae40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.354+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2ae8014070 con 0x7f2af8100940 2023-08-25T04:43:00.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.355+0000 7f2af659c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 0x7f2ae40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:00.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.355+0000 7f2af659c700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 0x7f2ae40a2410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2af8074410 tx=0x7f2ae0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:00.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.358+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2ae8051e00 con 0x7f2af8100940 2023-08-25T04:43:00.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.599+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2af8110a70 con 0x7f2af8100940 2023-08-25T04:43:00.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.599+0000 7f2aeffff700 1 -- 172.21.15.145:0/3659574318 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2ae80950c0 con 0x7f2af8100940 2023-08-25T04:43:00.601 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-25T04:43:00.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 msgr2=0x7f2ae40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:00.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 0x7f2ae40a2410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2af8074410 tx=0x7f2ae0009260 comp rx=0 tx=0).stop 2023-08-25T04:43:00.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 msgr2=0x7f2af8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:00.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800e9a0 tx=0x7f2ae800ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 shutdown_connections 2023-08-25T04:43:00.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f2ae409ff50 0x7f2ae40a2410 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2af8106940 0x7f2af80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2af8100940 0x7f2af8072c20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 --2- 172.21.15.145:0/3659574318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2af8101300 0x7f2af8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:00.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 >> 172.21.15.145:0/3659574318 conn(0x7f2af80fc410 msgr2=0x7f2af80fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:00.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 shutdown_connections 2023-08-25T04:43:00.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:00.602+0000 7f2afcb21700 1 -- 172.21.15.145:0/3659574318 wait complete. 2023-08-25T04:43:00.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:00 smithi145 ceph-mon[97681]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 966 B/s wr, 3 op/s 2023-08-25T04:43:00.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:00 smithi162 ceph-mon[130941]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 966 B/s wr, 3 op/s 2023-08-25T04:43:00.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:00 smithi183 ceph-mon[131414]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 966 B/s wr, 3 op/s 2023-08-25T04:43:01.111 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-25T04:43:01.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T04:43:01.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2023-08-25T04:43:01.112 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-25T04:43:01.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:01.113 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-25T04:43:01.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T04:43:01.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T04:43:01.113 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-25T04:43:01.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:01.114 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2023-08-25T04:43:01.160 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T04:43:01.160 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T04:43:01.160 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2023-08-25T04:43:01.161 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:43:01.162 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T04:43:01.163 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 41693sec preferred_lft 41693sec 2023-08-25T04:43:01.163 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::71eb:8a0d:636e:f0c3/64 scope link noprefixroute 2023-08-25T04:43:01.163 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:43:01.164 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T04:43:01.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:01.165 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:43:01.165 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:43:01.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add name ceph-brx type bridge 2023-08-25T04:43:01.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr flush dev ceph-brx 2023-08-25T04:43:01.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set ceph-brx up 2023-08-25T04:43:01.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T04:43:01.167 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:43:01.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:01.637 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T04:43:01.706 INFO:teuthology.orchestra.run.smithi145.stdout:1 2023-08-25T04:43:01.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:01.710 DEBUG:teuthology.orchestra.run.smithi145:> route 2023-08-25T04:43:01.824 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2023-08-25T04:43:01.824 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T04:43:01.824 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T04:43:01.824 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T04:43:01.825 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T04:43:01.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:01.826 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:43:01.827 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:43:01.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T04:43:01.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T04:43:01.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T04:43:01.828 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:43:01.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:01 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3659574318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:43:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:01 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3659574318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:43:01.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:01 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3659574318' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:43:02.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:02.578 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2023-08-25T04:43:02.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:02.624 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2023-08-25T04:43:02.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:02.708 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:43:02.708 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:43:02.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-25T04:43:02.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-25T04:43:02.709 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:43:02.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:02 smithi145 ceph-mon[97681]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-08-25T04:43:02.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-25T04:43:02.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:02.894 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:43:02.894 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:43:02.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-25T04:43:02.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T04:43:02.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-25T04:43:02.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-25T04:43:02.895 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-25T04:43:02.896 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:43:02.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:02 smithi162 ceph-mon[130941]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-08-25T04:43:02.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:02 smithi183 ceph-mon[131414]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-08-25T04:43:03.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:03.174 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:43:03.174 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:43:03.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2023-08-25T04:43:03.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T04:43:03.175 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:43:03.350 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-25T04:43:03.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T04:43:03.351 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:03.395 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-25T04:43:03.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T04:43:03.396 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:03.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2023-08-25T04:43:03.588 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T04:43:03.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T04:43:03.657 INFO:teuthology.orchestra.run.smithi145.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T04:43:03.660 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T04:43:03.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:03.661 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2023-08-25T04:43:03.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T04:43:03.706 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-25T04:43:03.750 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2023-08-25T04:43:03.820 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T04:43:03.820 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T04:43:03.888 INFO:teuthology.orchestra.run.smithi145.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T04:43:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T04:43:03.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:03.892 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2023-08-25T04:43:03.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi145.stderr:2023-08-25T04:43:03.956+0000 7fa131a385c0 -1 init, newargv = 0x563cf6c24350 newargc=15 2023-08-25T04:43:03.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi145.stderr:ceph-fuse[134890]: starting ceph client 2023-08-25T04:43:03.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi145.stderr:ceph-fuse[134890]: starting fuse 2023-08-25T04:43:05.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2023-08-25T04:43:05.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:04 smithi145 ceph-mon[97681]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2023-08-25T04:43:05.116 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T04:43:05.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T04:43:05.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:04 smithi162 ceph-mon[130941]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2023-08-25T04:43:05.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:04 smithi183 ceph-mon[131414]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2023-08-25T04:43:05.226 INFO:teuthology.orchestra.run.smithi145.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T04:43:05.229 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T04:43:05.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:05.230 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2023-08-25T04:43:05.274 INFO:teuthology.orchestra.run.smithi145.stdout:121 2023-08-25T04:43:05.274 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-08-25T04:43:05.275 DEBUG:teuthology.orchestra.run.smithi145:> sudo stdin-killer -- python3 -c ' 2023-08-25T04:43:05.275 DEBUG:teuthology.orchestra.run.smithi145:> import glob 2023-08-25T04:43:05.275 DEBUG:teuthology.orchestra.run.smithi145:> import re 2023-08-25T04:43:05.276 DEBUG:teuthology.orchestra.run.smithi145:> import os 2023-08-25T04:43:05.276 DEBUG:teuthology.orchestra.run.smithi145:> import subprocess 2023-08-25T04:43:05.276 DEBUG:teuthology.orchestra.run.smithi145:> 2023-08-25T04:43:05.276 DEBUG:teuthology.orchestra.run.smithi145:> def _find_admin_socket(client_name): 2023-08-25T04:43:05.277 DEBUG:teuthology.orchestra.run.smithi145:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-25T04:43:05.277 DEBUG:teuthology.orchestra.run.smithi145:> files = glob.glob(asok_path) 2023-08-25T04:43:05.277 DEBUG:teuthology.orchestra.run.smithi145:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-25T04:43:05.278 DEBUG:teuthology.orchestra.run.smithi145:> 2023-08-25T04:43:05.278 DEBUG:teuthology.orchestra.run.smithi145:> # Given a non-glob path, it better be there 2023-08-25T04:43:05.278 DEBUG:teuthology.orchestra.run.smithi145:> if "*" not in asok_path: 2023-08-25T04:43:05.279 DEBUG:teuthology.orchestra.run.smithi145:> assert(len(files) == 1) 2023-08-25T04:43:05.279 DEBUG:teuthology.orchestra.run.smithi145:> return files[0] 2023-08-25T04:43:05.279 DEBUG:teuthology.orchestra.run.smithi145:> 2023-08-25T04:43:05.279 DEBUG:teuthology.orchestra.run.smithi145:> for f in files: 2023-08-25T04:43:05.280 DEBUG:teuthology.orchestra.run.smithi145:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T04:43:05.280 DEBUG:teuthology.orchestra.run.smithi145:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T04:43:05.280 DEBUG:teuthology.orchestra.run.smithi145:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T04:43:05.280 DEBUG:teuthology.orchestra.run.smithi145:> contents = proc_f.read() 2023-08-25T04:43:05.281 DEBUG:teuthology.orchestra.run.smithi145:> if mountpoint in contents: 2023-08-25T04:43:05.281 DEBUG:teuthology.orchestra.run.smithi145:> return f 2023-08-25T04:43:05.281 DEBUG:teuthology.orchestra.run.smithi145:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T04:43:05.282 DEBUG:teuthology.orchestra.run.smithi145:> 2023-08-25T04:43:05.282 DEBUG:teuthology.orchestra.run.smithi145:> print(_find_admin_socket("client.admin")) 2023-08-25T04:43:05.282 DEBUG:teuthology.orchestra.run.smithi145:> ' 2023-08-25T04:43:05.456 INFO:teuthology.orchestra.run.smithi145.stdout:/var/run/ceph/ceph-client.admin.134890.asok 2023-08-25T04:43:05.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T04:43:05.470 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.134890.asok 2023-08-25T04:43:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:05.471 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.134890.asok status 2023-08-25T04:43:05.696 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:43:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "metadata": { 2023-08-25T04:43:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T04:43:05.696 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "entity_id": "admin", 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": "smithi145", 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "pid": "134890", 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "root": "/" 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T04:43:05.697 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_count": 0, 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_pinned_count": 0, 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24700, 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "inst": { 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "name": { 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "client", 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: "num": 24700 2023-08-25T04:43:05.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 1525523256 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T04:43:05.699 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2023-08-25T04:43:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2023-08-25T04:43:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T04:43:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 1525523256 2023-08-25T04:43:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T04:43:05.700 INFO:teuthology.orchestra.run.smithi145.stdout: "inst_str": "client.24700 172.21.15.145:0/1525523256", 2023-08-25T04:43:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "addr_str": "172.21.15.145:0/1525523256", 2023-08-25T04:43:05.701 INFO:teuthology.orchestra.run.smithi145.stdout: "inode_count": 1, 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_epoch": 17, 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch": 87, 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch_barrier": 0, 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "blocklisted": false, 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs" 2023-08-25T04:43:05.702 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:43:05.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:05.711 DEBUG:teuthology.orchestra.run.smithi145:> stat --file-system '--printf=%T 2023-08-25T04:43:05.712 DEBUG:teuthology.orchestra.run.smithi145:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:05.756 INFO:teuthology.orchestra.run.smithi145.stdout:fuseblk 2023-08-25T04:43:05.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:05.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:05.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:05.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:05.867 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-25T04:43:05.984 INFO:teuthology.orchestra.run.smithi145.stdout:# file: . 2023-08-25T04:43:05.985 INFO:teuthology.orchestra.run.smithi145.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-25T04:43:05.985 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:05.986 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:43:05.986 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T04:43:06.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:43:06.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.265+0000 7f3f407ff700 1 -- 172.21.15.145:0/2704013211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38100920 msgr2=0x7f3f38100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2704013211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38100920 0x7f3f38100d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f340099c0 tx=0x7f3f34009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 -- 172.21.15.145:0/2704013211 shutdown_connections 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2704013211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 0x7f3f38108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2704013211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38100920 0x7f3f38100d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2704013211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38101b10 0x7f3f38101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 -- 172.21.15.145:0/2704013211 >> 172.21.15.145:0/2704013211 conn(0x7f3f380fbe50 msgr2=0x7f3f380fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:06.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 -- 172.21.15.145:0/2704013211 shutdown_connections 2023-08-25T04:43:06.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.266+0000 7f3f407ff700 1 -- 172.21.15.145:0/2704013211 wait complete. 2023-08-25T04:43:06.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 Processor -- start 2023-08-25T04:43:06.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 -- start start 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 0x7f3f38198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 0x7f3f3819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a0390 con 0x7f3f38101b10 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a04d0 con 0x7f3f381024b0 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f407ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f381a0660 con 0x7f3f38100920 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f3dd9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 0x7f3f38198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f3e59b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f3ed9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 0x7f3f3819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f3e59b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60528/0 (socket says 172.21.15.145:60528) 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.267+0000 7f3f3e59b700 1 -- 172.21.15.145:0/2055726358 learned_addr learned my addr 172.21.15.145:0/2055726358 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 -- 172.21.15.145:0/2055726358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 msgr2=0x7f3f3819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 0x7f3f3819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 -- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 msgr2=0x7f3f38198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 0x7f3f38198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 -- 172.21.15.145:0/2055726358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f38100920 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3dd9a700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 0x7f3f38198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f3e59b700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f3400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401d070 con 0x7f3f38100920 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f34005050 con 0x7f3f38100920 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.268+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34004a70 con 0x7f3f38100920 2023-08-25T04:43:06.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.269+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f381a08e0 con 0x7f3f38100920 2023-08-25T04:43:06.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.269+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f381a0cb0 con 0x7f3f38100920 2023-08-25T04:43:06.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.270+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38192f70 con 0x7f3f38100920 2023-08-25T04:43:06.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.272+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3f34005450 con 0x7f3f38100920 2023-08-25T04:43:06.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.273+0000 7f3f2f7fe700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 0x7f3f240a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.273+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3f340cc650 con 0x7f3f38100920 2023-08-25T04:43:06.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.273+0000 7f3f3dd9a700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 0x7f3f240a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.274+0000 7f3f3dd9a700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 0x7f3f240a2460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f28011490 tx=0x7f3f28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:06.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.275+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3f34096220 con 0x7f3f38100920 2023-08-25T04:43:06.482 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.480+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f3810c180 con 0x7f3f38100920 2023-08-25T04:43:06.484 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.483+0000 7f3f2f7fe700 1 -- 172.21.15.145:0/2055726358 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f3f34099840 con 0x7f3f38100920 2023-08-25T04:43:06.485 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:06.485 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:06.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 msgr2=0x7f3f240a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 0x7f3f240a2460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f28011490 tx=0x7f3f28009260 comp rx=0 tx=0).stop 2023-08-25T04:43:06.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 msgr2=0x7f3f38198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f3400f690 comp rx=0 tx=0).stop 2023-08-25T04:43:06.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 shutdown_connections 2023-08-25T04:43:06.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3f2409ffa0 0x7f3f240a2460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f381024b0 0x7f3f3819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f38100920 0x7f3f38198810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 --2- 172.21.15.145:0/2055726358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f38101b10 0x7f3f38198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 >> 172.21.15.145:0/2055726358 conn(0x7f3f380fbe50 msgr2=0x7f3f38107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:06.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 shutdown_connections 2023-08-25T04:43:06.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.485+0000 7f3f407ff700 1 -- 172.21.15.145:0/2055726358 wait complete. 2023-08-25T04:43:06.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:43:06.508 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:43:06.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 -- 172.21.15.145:0/543167107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130101b00 msgr2=0x7fa130101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 --2- 172.21.15.145:0/543167107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130101b00 0x7fa130101f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099c0 tx=0x7fa118009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 -- 172.21.15.145:0/543167107 shutdown_connections 2023-08-25T04:43:06.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 --2- 172.21.15.145:0/543167107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 0x7fa130108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 --2- 172.21.15.145:0/543167107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130100910 0x7fa130100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 --2- 172.21.15.145:0/543167107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130101b00 0x7fa130101f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 -- 172.21.15.145:0/543167107 >> 172.21.15.145:0/543167107 conn(0x7fa1300fbe60 msgr2=0x7fa1300fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:06.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 -- 172.21.15.145:0/543167107 shutdown_connections 2023-08-25T04:43:06.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.708+0000 7fa134dff700 1 -- 172.21.15.145:0/543167107 wait complete. 2023-08-25T04:43:06.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 Processor -- start 2023-08-25T04:43:06.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 -- start start 2023-08-25T04:43:06.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 0x7fa130198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 0x7fa13019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301a0390 con 0x7fa130100910 2023-08-25T04:43:06.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.709+0000 7fa134dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301a04d0 con 0x7fa130101b00 2023-08-25T04:43:06.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa134dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1301a0660 con 0x7fa1301024a0 2023-08-25T04:43:06.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50400/0 (socket says 172.21.15.145:50400) 2023-08-25T04:43:06.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 -- 172.21.15.145:0/265377530 learned_addr learned my addr 172.21.15.145:0/265377530 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:06.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa127fff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 0x7fa130198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12ed9d700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 0x7fa13019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 -- 172.21.15.145:0/265377530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 msgr2=0x7fa13019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 0x7fa13019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 -- 172.21.15.145:0/265377530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 msgr2=0x7fa130198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 0x7fa130198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 -- 172.21.15.145:0/265377530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa130100910 2023-08-25T04:43:06.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa127fff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 0x7fa130198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:06.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.710+0000 7fa12e59c700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa12000d980 tx=0x7fa12000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:06.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.711+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120004020 con 0x7fa130100910 2023-08-25T04:43:06.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.711+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1301a0940 con 0x7fa130100910 2023-08-25T04:43:06.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.711+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12000f690 con 0x7fa130100910 2023-08-25T04:43:06.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.711+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120003a60 con 0x7fa130100910 2023-08-25T04:43:06.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.711+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1301a0ea0 con 0x7fa130100910 2023-08-25T04:43:06.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.712+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa130192f10 con 0x7fa130100910 2023-08-25T04:43:06.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.714+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa12000fb00 con 0x7fa130100910 2023-08-25T04:43:06.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.715+0000 7fa1277fe700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 0x7fa11c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:06.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.715+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa1200caf80 con 0x7fa130100910 2023-08-25T04:43:06.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.715+0000 7fa127fff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 0x7fa11c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:06.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.716+0000 7fa127fff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 0x7fa11c0a2490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa11800b580 tx=0x7fa11801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:06.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.720+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa120094ac0 con 0x7fa130100910 2023-08-25T04:43:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:06 smithi145 ceph-mon[97681]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-08-25T04:43:06.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:06 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2055726358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.923+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1301a0ad0 con 0x7fa130100910 2023-08-25T04:43:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.924+0000 7fa1277fe700 1 -- 172.21.15.145:0/265377530 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1301a0ad0 con 0x7fa130100910 2023-08-25T04:43:06.926 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:43:06.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 msgr2=0x7fa11c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 0x7fa11c0a2490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa11800b580 tx=0x7fa11801a040 comp rx=0 tx=0).stop 2023-08-25T04:43:06.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 msgr2=0x7fa130198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa12000d980 tx=0x7fa12000dc90 comp rx=0 tx=0).stop 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 shutdown_connections 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa11c09ffd0 0x7fa11c0a2490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1301024a0 0x7fa13019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa130100910 0x7fa130198810 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 --2- 172.21.15.145:0/265377530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa130101b00 0x7fa130198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 >> 172.21.15.145:0/265377530 conn(0x7fa1300fbe60 msgr2=0x7fa130107440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 shutdown_connections 2023-08-25T04:43:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:06.926+0000 7fa134dff700 1 -- 172.21.15.145:0/265377530 wait complete. 2023-08-25T04:43:06.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:43:07.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 -- 172.21.15.145:0/1897904451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 msgr2=0x7f296c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 --2- 172.21.15.145:0/1897904451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c101f80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f295c0099c0 tx=0x7f295c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 -- 172.21.15.145:0/1897904451 shutdown_connections 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 --2- 172.21.15.145:0/1897904451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 --2- 172.21.15.145:0/1897904451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 --2- 172.21.15.145:0/1897904451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c101f80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 -- 172.21.15.145:0/1897904451 >> 172.21.15.145:0/1897904451 conn(0x7f296c0fbea0 msgr2=0x7f296c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 -- 172.21.15.145:0/1897904451 shutdown_connections 2023-08-25T04:43:07.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.145+0000 7f29735ff700 1 -- 172.21.15.145:0/1897904451 wait complete. 2023-08-25T04:43:07.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 Processor -- start 2023-08-25T04:43:07.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 -- start start 2023-08-25T04:43:07.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.146+0000 7f29735ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0739f0 con 0x7f296c101b20 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f29735ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c073b60 con 0x7f296c1024c0 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f29735ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1a0650 con 0x7f296c100930 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50406/0 (socket says 172.21.15.145:50406) 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 -- 172.21.15.145:0/3807034027 learned_addr learned my addr 172.21.15.145:0/3807034027 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:07.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f297139b700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2971b9c700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 -- 172.21.15.145:0/3807034027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 msgr2=0x7f296c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 -- 172.21.15.145:0/3807034027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 msgr2=0x7f296c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 -- 172.21.15.145:0/3807034027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0095f0 con 0x7f296c101b20 2023-08-25T04:43:07.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.147+0000 7f2970b9a700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f295c009fd0 tx=0x7f295c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:07.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.148+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c01d070 con 0x7f296c101b20 2023-08-25T04:43:07.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.148+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c1a08d0 con 0x7f296c101b20 2023-08-25T04:43:07.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.148+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f295c005030 con 0x7f296c101b20 2023-08-25T04:43:07.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.148+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c004a90 con 0x7f296c101b20 2023-08-25T04:43:07.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.148+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1a0d50 con 0x7f296c101b20 2023-08-25T04:43:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.149+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c10c140 con 0x7f296c101b20 2023-08-25T04:43:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.151+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f295c0051a0 con 0x7f296c101b20 2023-08-25T04:43:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.152+0000 7f29627fc700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 0x7f29580a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.152+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f295c0cc1b0 con 0x7f296c101b20 2023-08-25T04:43:07.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.152+0000 7f297139b700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 0x7f29580a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.153+0000 7f297139b700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 0x7f29580a2490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f296c101980 tx=0x7f2968009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:07.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.156+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f295c095d80 con 0x7f296c101b20 2023-08-25T04:43:07.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:06 smithi162 ceph-mon[130941]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-08-25T04:43:07.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:06 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2055726358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:07.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:06 smithi183 ceph-mon[131414]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-08-25T04:43:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:06 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2055726358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:07.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.356+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f296c107830 con 0x7f296c101b20 2023-08-25T04:43:07.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.357+0000 7f29627fc700 1 -- 172.21.15.145:0/3807034027 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f295c0993a0 con 0x7f296c101b20 2023-08-25T04:43:07.359 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 msgr2=0x7f29580a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 0x7f29580a2490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f296c101980 tx=0x7f2968009260 comp rx=0 tx=0).stop 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 msgr2=0x7f296c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f295c009fd0 tx=0x7f295c009580 comp rx=0 tx=0).stop 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 shutdown_connections 2023-08-25T04:43:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f295809ffd0 0x7f29580a2490 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f296c1024c0 0x7f296c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c100930 0x7f296c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 --2- 172.21.15.145:0/3807034027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f296c101b20 0x7f296c072d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 >> 172.21.15.145:0/3807034027 conn(0x7f296c0fbea0 msgr2=0x7f296c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 shutdown_connections 2023-08-25T04:43:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.359+0000 7f29735ff700 1 -- 172.21.15.145:0/3807034027 wait complete. 2023-08-25T04:43:07.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:43:07.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 -- 172.21.15.145:0/3054478952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070108810 msgr2=0x7f007010ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 --2- 172.21.15.145:0/3054478952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070108810 0x7f007010ac80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f006800b0a0 tx=0x7f006800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 -- 172.21.15.145:0/3054478952 shutdown_connections 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 --2- 172.21.15.145:0/3054478952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070108810 0x7f007010ac80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 --2- 172.21.15.145:0/3054478952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 0x7f0070072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 --2- 172.21.15.145:0/3054478952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070073320 0x7f00700737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.577+0000 7f0076bff700 1 -- 172.21.15.145:0/3054478952 >> 172.21.15.145:0/3054478952 conn(0x7f00700fbed0 msgr2=0x7f00700fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.578+0000 7f0076bff700 1 -- 172.21.15.145:0/3054478952 shutdown_connections 2023-08-25T04:43:07.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.578+0000 7f0076bff700 1 -- 172.21.15.145:0/3054478952 wait complete. 2023-08-25T04:43:07.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 Processor -- start 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 -- start start 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 0x7f0070198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 0x7f007019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00701a0420 con 0x7f0070073320 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00701a0560 con 0x7f0070108810 2023-08-25T04:43:07.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f0076bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00701a06f0 con 0x7f0070074900 2023-08-25T04:43:07.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f007499b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f007499b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50412/0 (socket says 172.21.15.145:50412) 2023-08-25T04:43:07.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f007499b700 1 -- 172.21.15.145:0/2475397514 learned_addr learned my addr 172.21.15.145:0/2475397514 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f007519c700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 0x7f007019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.579+0000 7f006ffff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 0x7f0070198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 -- 172.21.15.145:0/2475397514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 msgr2=0x7f0070198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 0x7f0070198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 -- 172.21.15.145:0/2475397514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 msgr2=0x7f007019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 0x7f007019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 -- 172.21.15.145:0/2475397514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068009d00 con 0x7f0070073320 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007519c700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 0x7f007019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f007499b700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f006400e9a0 tx=0x7f006400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064009790 con 0x7f0070073320 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0064010430 con 0x7f0070073320 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00701a09d0 con 0x7f0070073320 2023-08-25T04:43:07.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064018680 con 0x7f0070073320 2023-08-25T04:43:07.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.580+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00701a0ee0 con 0x7f0070073320 2023-08-25T04:43:07.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.581+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0070106f10 con 0x7f0070073320 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.585+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f00640105a0 con 0x7f0070073320 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.586+0000 7f006dffb700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 0x7f00600a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.586+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0064014070 con 0x7f0070073320 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.586+0000 7f006ffff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 0x7f00600a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.587+0000 7f006ffff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 0x7f00600a2410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f005c009c60 tx=0x7f005c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:07.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.588+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0064095360 con 0x7f0070073320 2023-08-25T04:43:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.793+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f007010e530 con 0x7f0070073320 2023-08-25T04:43:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.794+0000 7f006dffb700 1 -- 172.21.15.145:0/2475397514 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00640260a0 con 0x7f0070073320 2023-08-25T04:43:07.796 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:43:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 msgr2=0x7f00600a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 0x7f00600a2410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f005c009c60 tx=0x7f005c009260 comp rx=0 tx=0).stop 2023-08-25T04:43:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 msgr2=0x7f00701988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f006400e9a0 tx=0x7f006400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 shutdown_connections 2023-08-25T04:43:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f006009ff50 0x7f00600a2410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0070108810 0x7f007019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0070073320 0x7f00701988a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 --2- 172.21.15.145:0/2475397514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070074900 0x7f0070198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:07.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 >> 172.21.15.145:0/2475397514 conn(0x7f00700fbed0 msgr2=0x7f00700fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:07.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 shutdown_connections 2023-08-25T04:43:07.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:07.797+0000 7f0076bff700 1 -- 172.21.15.145:0/2475397514 wait complete. 2023-08-25T04:43:07.816 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:43:08.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 -- 172.21.15.145:0/415645565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d78100910 msgr2=0x7f3d78100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 --2- 172.21.15.145:0/415645565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d78100910 0x7f3d78100d30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099c0 tx=0x7f3d74009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 -- 172.21.15.145:0/415645565 shutdown_connections 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 --2- 172.21.15.145:0/415645565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d781024a0 0x7f3d78108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 --2- 172.21.15.145:0/415645565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d78100910 0x7f3d78100d30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 --2- 172.21.15.145:0/415645565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78101b00 0x7f3d78101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 -- 172.21.15.145:0/415645565 >> 172.21.15.145:0/415645565 conn(0x7f3d780fbe60 msgr2=0x7f3d780fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 -- 172.21.15.145:0/415645565 shutdown_connections 2023-08-25T04:43:08.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.015+0000 7f3d807ff700 1 -- 172.21.15.145:0/415645565 wait complete. 2023-08-25T04:43:08.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.016+0000 7f3d807ff700 1 Processor -- start 2023-08-25T04:43:08.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.016+0000 7f3d807ff700 1 -- start start 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78101b00 0x7f3d78072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780739f0 con 0x7f3d781024a0 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78073b60 con 0x7f3d78100910 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d807ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d781a0670 con 0x7f3d78101b00 2023-08-25T04:43:08.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7e59b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7ed9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7ed9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50430/0 (socket says 172.21.15.145:50430) 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7e59b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:36486/0 (socket says 172.21.15.145:36486) 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7ed9c700 1 -- 172.21.15.145:0/559019471 learned_addr learned my addr 172.21.15.145:0/559019471 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7dd9a700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78101b00 0x7f3d78072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7ed9c700 1 -- 172.21.15.145:0/559019471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78101b00 msgr2=0x7f3d78072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.017+0000 7f3d7ed9c700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78101b00 0x7f3d78072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d7ed9c700 1 -- 172.21.15.145:0/559019471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 msgr2=0x7f3d780746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d7ed9c700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d7ed9c700 1 -- 172.21.15.145:0/559019471 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740095f0 con 0x7f3d781024a0 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d7e59b700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d7ed9c700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000ef30 tx=0x7f3d7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70018070 con 0x7f3d781024a0 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d7000f040 con 0x7f3d781024a0 2023-08-25T04:43:08.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d781a08f0 con 0x7f3d781024a0 2023-08-25T04:43:08.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000cd00 con 0x7f3d781024a0 2023-08-25T04:43:08.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.018+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d781a0e00 con 0x7f3d781024a0 2023-08-25T04:43:08.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.019+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7810c120 con 0x7f3d781024a0 2023-08-25T04:43:08.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.021+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3d70003a00 con 0x7f3d781024a0 2023-08-25T04:43:08.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.023+0000 7f3d6f7fe700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 0x7f3d640a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.023+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3d700caf70 con 0x7f3d781024a0 2023-08-25T04:43:08.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.023+0000 7f3d7e59b700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 0x7f3d640a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.024+0000 7f3d7e59b700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 0x7f3d640a2490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009990 tx=0x7f3d7401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.028+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3d70094b40 con 0x7f3d781024a0 2023-08-25T04:43:08.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:07 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/265377530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:07 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3807034027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:07 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/265377530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:07 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3807034027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:07 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/265377530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:07 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3807034027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:08.230 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.228+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3d78107850 con 0x7f3d781024a0 2023-08-25T04:43:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.229+0000 7f3d6f7fe700 1 -- 172.21.15.145:0/559019471 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3d70098160 con 0x7f3d781024a0 2023-08-25T04:43:08.231 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:43:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 msgr2=0x7f3d640a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 0x7f3d640a2490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009990 tx=0x7f3d7401a040 comp rx=0 tx=0).stop 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 msgr2=0x7f3d78073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000ef30 tx=0x7f3d7000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 shutdown_connections 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3d6409ffd0 0x7f3d640a2490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d781024a0 0x7f3d78073250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d78100910 0x7f3d780746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 --2- 172.21.15.145:0/559019471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78101b00 0x7f3d78072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.231+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 >> 172.21.15.145:0/559019471 conn(0x7f3d780fbe60 msgr2=0x7f3d780fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.232+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 shutdown_connections 2023-08-25T04:43:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.232+0000 7f3d807ff700 1 -- 172.21.15.145:0/559019471 wait complete. 2023-08-25T04:43:08.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T04:43:08.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 -- 172.21.15.145:0/1061904694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f8101b00 msgr2=0x7f92f8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/1061904694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f8101b00 0x7f92f8101f60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0099c0 tx=0x7f92ec009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 -- 172.21.15.145:0/1061904694 shutdown_connections 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/1061904694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f81024a0 0x7f92f8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/1061904694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8100910 0x7f92f8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/1061904694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f8101b00 0x7f92f8101f60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 -- 172.21.15.145:0/1061904694 >> 172.21.15.145:0/1061904694 conn(0x7f92f80fbe60 msgr2=0x7f92f80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:08.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 -- 172.21.15.145:0/1061904694 shutdown_connections 2023-08-25T04:43:08.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.449+0000 7f92fd9ff700 1 -- 172.21.15.145:0/1061904694 wait complete. 2023-08-25T04:43:08.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.450+0000 7f92fd9ff700 1 Processor -- start 2023-08-25T04:43:08.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 -- start start 2023-08-25T04:43:08.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 0x7f92f80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 0x7f92f8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80739f0 con 0x7f92f81024a0 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8073b60 con 0x7f92f8100910 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92fd9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81a06b0 con 0x7f92f8101b00 2023-08-25T04:43:08.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.451+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 0x7f92f8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50454/0 (socket says 172.21.15.145:50454) 2023-08-25T04:43:08.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f77fe700 1 -- 172.21.15.145:0/2541487201 learned_addr learned my addr 172.21.15.145:0/2541487201 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:08.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f6ffd700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 0x7f92f80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 -- 172.21.15.145:0/2541487201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 msgr2=0x7f92f80746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 0x7f92f80746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 -- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 msgr2=0x7f92f8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 -- 172.21.15.145:0/2541487201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0095f0 con 0x7f92f8101b00 2023-08-25T04:43:08.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f6ffd700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 0x7f92f80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:08.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f77fe700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:08.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92f67fc700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 0x7f92f8072d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098e0 tx=0x7f92ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.452+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec01d070 con 0x7f92f8101b00 2023-08-25T04:43:08.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.453+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ec005020 con 0x7f92f8101b00 2023-08-25T04:43:08.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.453+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f81a0930 con 0x7f92f8101b00 2023-08-25T04:43:08.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.453+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004a70 con 0x7f92f8101b00 2023-08-25T04:43:08.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.453+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81a0cf0 con 0x7f92f8101b00 2023-08-25T04:43:08.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.454+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f810c170 con 0x7f92f8101b00 2023-08-25T04:43:08.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.456+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f92ec005400 con 0x7f92f8101b00 2023-08-25T04:43:08.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.457+0000 7f92fc9fd700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 0x7f92e00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.457+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f92ec0cbfa0 con 0x7f92f8101b00 2023-08-25T04:43:08.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.457+0000 7f92f6ffd700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 0x7f92e00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.458+0000 7f92f6ffd700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 0x7f92e00a2410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92f8101960 tx=0x7f92e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.459+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f92ec095b70 con 0x7f92f8101b00 2023-08-25T04:43:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.674+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92f8107840 con 0x7f92f8101b00 2023-08-25T04:43:08.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.676+0000 7f92fc9fd700 1 -- 172.21.15.145:0/2541487201 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+600 (secure 0 0 0) 0x7f92ec005190 con 0x7f92f8101b00 2023-08-25T04:43:08.677 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/4198991683 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1908809275 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/227013713 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/417312367 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1961008949 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/3102359690 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:08.678 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:08.679 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:08.679 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.677+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 msgr2=0x7f92e00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.677+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 0x7f92e00a2410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92f8101960 tx=0x7f92e8009260 comp rx=0 tx=0).stop 2023-08-25T04:43:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 msgr2=0x7f92f8072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 0x7f92f8072d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098e0 tx=0x7f92ec00f690 comp rx=0 tx=0).stop 2023-08-25T04:43:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 shutdown_connections 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f92e009ff50 0x7f92e00a2410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92f81024a0 0x7f92f8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f8100910 0x7f92f80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 --2- 172.21.15.145:0/2541487201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92f8101b00 0x7f92f8072d10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 >> 172.21.15.145:0/2541487201 conn(0x7f92f80fbe60 msgr2=0x7f92f80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 shutdown_connections 2023-08-25T04:43:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.678+0000 7f92fd9ff700 1 -- 172.21.15.145:0/2541487201 wait complete. 2023-08-25T04:43:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:listed 10 entries 2023-08-25T04:43:08.698 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:43:08.699 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T04:43:08.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:43:08.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 -- 172.21.15.145:0/1138679589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec100930 msgr2=0x7fcbec100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/1138679589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec100930 0x7fcbec100d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009990 tx=0x7fcbe8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 -- 172.21.15.145:0/1138679589 shutdown_connections 2023-08-25T04:43:08.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/1138679589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec1024c0 0x7fcbec108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/1138679589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec100930 0x7fcbec100d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/1138679589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec101b20 0x7fcbec101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 -- 172.21.15.145:0/1138679589 >> 172.21.15.145:0/1138679589 conn(0x7fcbec0fbe60 msgr2=0x7fcbec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:08.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 -- 172.21.15.145:0/1138679589 shutdown_connections 2023-08-25T04:43:08.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.982+0000 7fcbf39ff700 1 -- 172.21.15.145:0/1138679589 wait complete. 2023-08-25T04:43:08.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.983+0000 7fcbf39ff700 1 Processor -- start 2023-08-25T04:43:08.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.983+0000 7fcbf39ff700 1 -- start start 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 0x7fcbec198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec101b20 0x7fcbec198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1a03a0 con 0x7fcbec100930 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1a04e0 con 0x7fcbec1024c0 2023-08-25T04:43:08.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf39ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1a0670 con 0x7fcbec101b20 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf179b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 0x7fcbec198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:36520/0 (socket says 172.21.15.145:36520) 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 -- 172.21.15.145:0/980191102 learned_addr learned my addr 172.21.15.145:0/980191102 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf0f9a700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec101b20 0x7fcbec198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 -- 172.21.15.145:0/980191102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec101b20 msgr2=0x7fcbec198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec101b20 0x7fcbec198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 -- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 msgr2=0x7fcbec198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 0x7fcbec198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf1f9c700 1 -- 172.21.15.145:0/980191102 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80095f0 con 0x7fcbec1024c0 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.984+0000 7fcbf179b700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 0x7fcbec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbf1f9c700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400ef30 tx=0x7fcbe400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4018070 con 0x7fcbec1024c0 2023-08-25T04:43:08.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec1a08f0 con 0x7fcbec1024c0 2023-08-25T04:43:08.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbe400f040 con 0x7fcbec1024c0 2023-08-25T04:43:08.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400cd00 con 0x7fcbec1024c0 2023-08-25T04:43:08.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.985+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec1a0e00 con 0x7fcbec1024c0 2023-08-25T04:43:08.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.986+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec10c190 con 0x7fcbec1024c0 2023-08-25T04:43:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.988+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcbe4003a00 con 0x7fcbec1024c0 2023-08-25T04:43:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.989+0000 7fcbe27fc700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 0x7fcbd80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.989+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcbe40cb3b0 con 0x7fcbec1024c0 2023-08-25T04:43:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.989+0000 7fcbf179b700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 0x7fcbd80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:08.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.990+0000 7fcbf179b700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 0x7fcbd80a2460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800b2e0 tx=0x7fcbe8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:08.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:08.991+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcbe4094f80 con 0x7fcbec1024c0 2023-08-25T04:43:09.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:08 smithi145 ceph-mon[97681]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-25T04:43:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2475397514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/559019471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2541487201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:08 smithi162 ceph-mon[130941]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-25T04:43:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2475397514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/559019471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2541487201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:08 smithi183 ceph-mon[131414]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-08-25T04:43:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2475397514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/559019471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:09.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2541487201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:09.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.193+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbec10c380 con 0x7fcbec1024c0 2023-08-25T04:43:09.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.195+0000 7fcbe27fc700 1 -- 172.21.15.145:0/980191102 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7fcbe40985a0 con 0x7fcbec1024c0 2023-08-25T04:43:09.200 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:09.200 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:09.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 msgr2=0x7fcbd80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 0x7fcbd80a2460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800b2e0 tx=0x7fcbe8005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 msgr2=0x7fcbec19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400ef30 tx=0x7fcbe400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 shutdown_connections 2023-08-25T04:43:09.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcbd809ffa0 0x7fcbd80a2460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbec1024c0 0x7fcbec19fdd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbec100930 0x7fcbec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 --2- 172.21.15.145:0/980191102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbec101b20 0x7fcbec198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 >> 172.21.15.145:0/980191102 conn(0x7fcbec0fbe60 msgr2=0x7fcbec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:09.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 shutdown_connections 2023-08-25T04:43:09.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.197+0000 7fcbf39ff700 1 -- 172.21.15.145:0/980191102 wait complete. 2023-08-25T04:43:09.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:43:09.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:43:09.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 -- 172.21.15.145:0/3718206794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce8100930 msgr2=0x7efce8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 --2- 172.21.15.145:0/3718206794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce8100930 0x7efce8100d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0099c0 tx=0x7efcdc009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 -- 172.21.15.145:0/3718206794 shutdown_connections 2023-08-25T04:43:09.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 --2- 172.21.15.145:0/3718206794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce81024c0 0x7efce8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 --2- 172.21.15.145:0/3718206794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce8100930 0x7efce8100d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 --2- 172.21.15.145:0/3718206794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 -- 172.21.15.145:0/3718206794 >> 172.21.15.145:0/3718206794 conn(0x7efce80fbea0 msgr2=0x7efce80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 -- 172.21.15.145:0/3718206794 shutdown_connections 2023-08-25T04:43:09.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.416+0000 7efceebff700 1 -- 172.21.15.145:0/3718206794 wait complete. 2023-08-25T04:43:09.420 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 Processor -- start 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 -- start start 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce8100930 0x7efce80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.417+0000 7efceebff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce80739f0 con 0x7efce81024c0 2023-08-25T04:43:09.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efceebff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce8073b60 con 0x7efce8101b20 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efceebff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce81a0690 con 0x7efce8100930 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50486/0 (socket says 172.21.15.145:50486) 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 -- 172.21.15.145:0/502622212 learned_addr learned my addr 172.21.15.145:0/502622212 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efce7fff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efcec99b700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce8100930 0x7efce80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 -- 172.21.15.145:0/502622212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce8100930 msgr2=0x7efce80746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce8100930 0x7efce80746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 -- 172.21.15.145:0/502622212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 msgr2=0x7efce8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 -- 172.21.15.145:0/502622212 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0009c40 con 0x7efce81024c0 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efce7fff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.418+0000 7efced19c700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efce000c370 tx=0x7efce000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.419+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0017070 con 0x7efce81024c0 2023-08-25T04:43:09.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.419+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efce000f040 con 0x7efce81024c0 2023-08-25T04:43:09.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.419+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc0095f0 con 0x7efce81024c0 2023-08-25T04:43:09.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.419+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce000e050 con 0x7efce81024c0 2023-08-25T04:43:09.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.419+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce81a0c90 con 0x7efce81024c0 2023-08-25T04:43:09.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.420+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce8107f40 con 0x7efce81024c0 2023-08-25T04:43:09.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.422+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7efce000cb60 con 0x7efce81024c0 2023-08-25T04:43:09.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.424+0000 7efce5ffb700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 0x7efcd80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.424+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efce00cad40 con 0x7efce81024c0 2023-08-25T04:43:09.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.424+0000 7efcec99b700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 0x7efcd80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.425+0000 7efcec99b700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 0x7efcd80a2490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009fd0 tx=0x7efcdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:09.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.428+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7efce0094880 con 0x7efce81024c0 2023-08-25T04:43:09.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.632+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efce802cdc0 con 0x7efce81024c0 2023-08-25T04:43:09.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.632+0000 7efce5ffb700 1 -- 172.21.15.145:0/502622212 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7efce0097ea0 con 0x7efce81024c0 2023-08-25T04:43:09.635 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:43:09.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.634+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 msgr2=0x7efcd80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 0x7efcd80a2490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009fd0 tx=0x7efcdc005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 msgr2=0x7efce8073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efce000c370 tx=0x7efce000c680 comp rx=0 tx=0).stop 2023-08-25T04:43:09.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 shutdown_connections 2023-08-25T04:43:09.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efcd809ffd0 0x7efcd80a2490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efce81024c0 0x7efce8073250 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efce8100930 0x7efce80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 --2- 172.21.15.145:0/502622212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce8101b20 0x7efce8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 >> 172.21.15.145:0/502622212 conn(0x7efce80fbea0 msgr2=0x7efce8105080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:09.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 shutdown_connections 2023-08-25T04:43:09.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.635+0000 7efceebff700 1 -- 172.21.15.145:0/502622212 wait complete. 2023-08-25T04:43:09.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:43:09.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 -- 172.21.15.145:0/796712297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a0100930 msgr2=0x7ff9a0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/796712297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a0100930 0x7ff9a0100d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009990 tx=0x7ff99c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 -- 172.21.15.145:0/796712297 shutdown_connections 2023-08-25T04:43:09.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/796712297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a01024c0 0x7ff9a0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/796712297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a0100930 0x7ff9a0100d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/796712297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 -- 172.21.15.145:0/796712297 >> 172.21.15.145:0/796712297 conn(0x7ff9a00fbea0 msgr2=0x7ff9a00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:09.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 -- 172.21.15.145:0/796712297 shutdown_connections 2023-08-25T04:43:09.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.856+0000 7ff9a81ff700 1 -- 172.21.15.145:0/796712297 wait complete. 2023-08-25T04:43:09.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.857+0000 7ff9a81ff700 1 Processor -- start 2023-08-25T04:43:09.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.857+0000 7ff9a81ff700 1 -- start start 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.857+0000 7ff9a81ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.857+0000 7ff9a81ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a81ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a01024c0 0x7ff9a019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a81ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a01a03e0 con 0x7ff9a0101b20 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a81ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a01a0520 con 0x7ff9a0100930 2023-08-25T04:43:09.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a81ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a01a06b0 con 0x7ff9a01024c0 2023-08-25T04:43:09.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a5f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a5f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:36550/0 (socket says 172.21.15.145:36550) 2023-08-25T04:43:09.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a5f9b700 1 -- 172.21.15.145:0/3910905937 learned_addr learned my addr 172.21.15.145:0/3910905937 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:09.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a679c700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a01024c0 0x7ff9a019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 -- 172.21.15.145:0/3910905937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a01024c0 msgr2=0x7ff9a019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a01024c0 0x7ff9a019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 -- 172.21.15.145:0/3910905937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 msgr2=0x7ff9a0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:09.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:09.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 -- 172.21.15.145:0/3910905937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0095f0 con 0x7ff9a0101b20 2023-08-25T04:43:09.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a5f9b700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:09.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.858+0000 7ff9a579a700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0198da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff99000e9f0 tx=0x7ff99000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:09.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.859+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000c9c0 con 0x7ff9a0101b20 2023-08-25T04:43:09.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.859+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a01a0990 con 0x7ff9a0101b20 2023-08-25T04:43:09.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.859+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff990010430 con 0x7ff9a0101b20 2023-08-25T04:43:09.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.859+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990018680 con 0x7ff9a0101b20 2023-08-25T04:43:09.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.859+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a01a0ef0 con 0x7ff9a0101b20 2023-08-25T04:43:09.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.860+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0192f80 con 0x7ff9a0101b20 2023-08-25T04:43:09.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.862+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ff990010860 con 0x7ff9a0101b20 2023-08-25T04:43:09.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.863+0000 7ff996ffd700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 0x7ff98c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.863+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff990014070 con 0x7ff9a0101b20 2023-08-25T04:43:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.863+0000 7ff9a5f9b700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 0x7ff98c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.864+0000 7ff9a5f9b700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 0x7ff98c0a2490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b2e0 tx=0x7ff99c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:09.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:09.867+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff990051e00 con 0x7ff9a0101b20 2023-08-25T04:43:10.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.066+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9a002cd00 con 0x7ff9a0101b20 2023-08-25T04:43:10.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.067+0000 7ff996ffd700 1 -- 172.21.15.145:0/3910905937 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9900950e0 con 0x7ff9a0101b20 2023-08-25T04:43:10.069 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:43:10.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.069+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 msgr2=0x7ff98c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.069+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 0x7ff98c0a2490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00b2e0 tx=0x7ff99c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.069+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 msgr2=0x7ff9a0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0198da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff99000e9f0 tx=0x7ff99000ed00 comp rx=0 tx=0).stop 2023-08-25T04:43:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 shutdown_connections 2023-08-25T04:43:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff98c09ffd0 0x7ff98c0a2490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a01024c0 0x7ff9a019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a0100930 0x7ff9a0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 --2- 172.21.15.145:0/3910905937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a0101b20 0x7ff9a0198da0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 >> 172.21.15.145:0/3910905937 conn(0x7ff9a00fbea0 msgr2=0x7ff9a0107450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:10.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 shutdown_connections 2023-08-25T04:43:10.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.070+0000 7ff9a81ff700 1 -- 172.21.15.145:0/3910905937 wait complete. 2023-08-25T04:43:10.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:09 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/980191102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:09 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/502622212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:10.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:43:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:09 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/980191102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:10.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:09 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/502622212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:09 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/980191102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:09 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/502622212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:43:10.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 -- 172.21.15.145:0/4127580233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588074960 msgr2=0x7fd588072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/4127580233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588074960 0x7fd588072db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd584009990 tx=0x7fd584009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 -- 172.21.15.145:0/4127580233 shutdown_connections 2023-08-25T04:43:10.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/4127580233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5881087f0 0x7fd58810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/4127580233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588074960 0x7fd588072db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/4127580233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd588073380 0x7fd588073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 -- 172.21.15.145:0/4127580233 >> 172.21.15.145:0/4127580233 conn(0x7fd5880fbea0 msgr2=0x7fd5880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 -- 172.21.15.145:0/4127580233 shutdown_connections 2023-08-25T04:43:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.290+0000 7fd58f7ff700 1 -- 172.21.15.145:0/4127580233 wait complete. 2023-08-25T04:43:10.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 Processor -- start 2023-08-25T04:43:10.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 -- start start 2023-08-25T04:43:10.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 0x7fd588198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881087f0 0x7fd58819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a0420 con 0x7fd588073380 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a0560 con 0x7fd588074960 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58f7ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a06f0 con 0x7fd5881087f0 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.291+0000 7fd58d59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50522/0 (socket says 172.21.15.145:50522) 2023-08-25T04:43:10.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 -- 172.21.15.145:0/1759635292 learned_addr learned my addr 172.21.15.145:0/1759635292 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:10.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58cd9a700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 0x7fd588198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58dd9c700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881087f0 0x7fd58819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 -- 172.21.15.145:0/1759635292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881087f0 msgr2=0x7fd58819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881087f0 0x7fd58819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 -- 172.21.15.145:0/1759635292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 msgr2=0x7fd588198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 0x7fd588198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 -- 172.21.15.145:0/1759635292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580009c40 con 0x7fd588073380 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58cd9a700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 0x7fd588198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58d59b700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd584004730 tx=0x7fd584005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58401d070 con 0x7fd588073380 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5840095f0 con 0x7fd588073380 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd584004cd0 con 0x7fd588073380 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.292+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584005610 con 0x7fd588073380 2023-08-25T04:43:10.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.293+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5881a0ba0 con 0x7fd588073380 2023-08-25T04:43:10.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.294+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588106ea0 con 0x7fd588073380 2023-08-25T04:43:10.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.295+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd58400fc70 con 0x7fd588073380 2023-08-25T04:43:10.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.297+0000 7fd57e7fc700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 0x7fd5740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.297+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5840cc8f0 con 0x7fd588073380 2023-08-25T04:43:10.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.297+0000 7fd58cd9a700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 0x7fd5740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.297+0000 7fd58cd9a700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 0x7fd5740a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd578005950 tx=0x7fd5780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:10.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.300+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd58405ae00 con 0x7fd588073380 2023-08-25T04:43:10.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.503+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd58802cd30 con 0x7fd588073380 2023-08-25T04:43:10.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.503+0000 7fd57e7fc700 1 -- 172.21.15.145:0/1759635292 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5840964c0 con 0x7fd588073380 2023-08-25T04:43:10.505 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:43:10.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 msgr2=0x7fd5740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 0x7fd5740a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd578005950 tx=0x7fd5780058e0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 msgr2=0x7fd5881988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd584004730 tx=0x7fd584005ef0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 shutdown_connections 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd57409ff50 0x7fd5740a2410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881087f0 0x7fd58819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd588073380 0x7fd5881988a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 --2- 172.21.15.145:0/1759635292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd588074960 0x7fd588198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 >> 172.21.15.145:0/1759635292 conn(0x7fd5880fbea0 msgr2=0x7fd5880fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:10.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 shutdown_connections 2023-08-25T04:43:10.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.506+0000 7fd58f7ff700 1 -- 172.21.15.145:0/1759635292 wait complete. 2023-08-25T04:43:10.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:43:10.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 -- 172.21.15.145:0/175274432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c101b20 msgr2=0x7f256c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 --2- 172.21.15.145:0/175274432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c101b20 0x7f256c101f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25580099c0 tx=0x7f2558009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 -- 172.21.15.145:0/175274432 shutdown_connections 2023-08-25T04:43:10.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 --2- 172.21.15.145:0/175274432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c1024c0 0x7f256c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 --2- 172.21.15.145:0/175274432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c100930 0x7f256c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 --2- 172.21.15.145:0/175274432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c101b20 0x7f256c101f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 -- 172.21.15.145:0/175274432 >> 172.21.15.145:0/175274432 conn(0x7f256c0fbea0 msgr2=0x7f256c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:10.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 -- 172.21.15.145:0/175274432 shutdown_connections 2023-08-25T04:43:10.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.729+0000 7f25731ff700 1 -- 172.21.15.145:0/175274432 wait complete. 2023-08-25T04:43:10.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.730+0000 7f25731ff700 1 Processor -- start 2023-08-25T04:43:10.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.730+0000 7f25731ff700 1 -- start start 2023-08-25T04:43:10.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.730+0000 7f25731ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f25731ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c101b20 0x7f256c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f25731ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c1024c0 0x7f256c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f25731ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1a0380 con 0x7f256c100930 2023-08-25T04:43:10.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f25731ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1a04c0 con 0x7f256c101b20 2023-08-25T04:43:10.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f25731ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1a0650 con 0x7f256c1024c0 2023-08-25T04:43:10.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50538/0 (socket says 172.21.15.145:50538) 2023-08-25T04:43:10.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 -- 172.21.15.145:0/231623774 learned_addr learned my addr 172.21.15.145:0/231623774 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:10.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f256bfff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c101b20 0x7f256c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f257179c700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c1024c0 0x7f256c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 -- 172.21.15.145:0/231623774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c1024c0 msgr2=0x7f256c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c1024c0 0x7f256c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 -- 172.21.15.145:0/231623774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c101b20 msgr2=0x7f256c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c101b20 0x7f256c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 -- 172.21.15.145:0/231623774 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25580095f0 con 0x7f256c100930 2023-08-25T04:43:10.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.731+0000 7f2570f9b700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f256000e970 tx=0x7f256000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:10.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.732+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560009790 con 0x7f256c100930 2023-08-25T04:43:10.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.732+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f256000eec0 con 0x7f256c100930 2023-08-25T04:43:10.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.732+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c1a0930 con 0x7f256c100930 2023-08-25T04:43:10.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.732+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560003cf0 con 0x7f256c100930 2023-08-25T04:43:10.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.732+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1a0e40 con 0x7f256c100930 2023-08-25T04:43:10.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.733+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c10c140 con 0x7f256c100930 2023-08-25T04:43:10.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.735+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f256000ccd0 con 0x7f256c100930 2023-08-25T04:43:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.736+0000 7f2569ffb700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 0x7f255c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.736+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2560014070 con 0x7f256c100930 2023-08-25T04:43:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.737+0000 7f256bfff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 0x7f255c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.737+0000 7f256bfff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 0x7f255c0a2490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f255800b580 tx=0x7f2558005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.740+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2560095230 con 0x7f256c100930 2023-08-25T04:43:10.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:10 smithi145 ceph-mon[97681]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3910905937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:10.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1759635292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:10.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.945+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f256c1a1090 con 0x7f256c100930 2023-08-25T04:43:10.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.946+0000 7f2569ffb700 1 -- 172.21.15.145:0/231623774 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2560098850 con 0x7f256c100930 2023-08-25T04:43:10.948 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:43:10.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 msgr2=0x7f255c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 0x7f255c0a2490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f255800b580 tx=0x7f2558005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 msgr2=0x7f256c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f256000e970 tx=0x7f256000ec80 comp rx=0 tx=0).stop 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 shutdown_connections 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f255c09ffd0 0x7f255c0a2490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256c1024c0 0x7f256c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f256c100930 0x7f256c198800 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 --2- 172.21.15.145:0/231623774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f256c101b20 0x7f256c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 >> 172.21.15.145:0/231623774 conn(0x7f256c0fbea0 msgr2=0x7f256c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:10.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 shutdown_connections 2023-08-25T04:43:10.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:10.948+0000 7f25731ff700 1 -- 172.21.15.145:0/231623774 wait complete. 2023-08-25T04:43:10.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T04:43:11.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:10 smithi162 ceph-mon[130941]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3910905937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1759635292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:11.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 -- 172.21.15.145:0/116578384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 msgr2=0x7f7648101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 --2- 172.21.15.145:0/116578384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648101f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f763c009920 tx=0x7f763c009c30 comp rx=0 tx=0).stop 2023-08-25T04:43:11.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 -- 172.21.15.145:0/116578384 shutdown_connections 2023-08-25T04:43:11.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 --2- 172.21.15.145:0/116578384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76481024c0 0x7f7648108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 --2- 172.21.15.145:0/116578384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7648100930 0x7f7648100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 --2- 172.21.15.145:0/116578384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648101f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 -- 172.21.15.145:0/116578384 >> 172.21.15.145:0/116578384 conn(0x7f76480fbea0 msgr2=0x7f76480fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:11.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 -- 172.21.15.145:0/116578384 shutdown_connections 2023-08-25T04:43:11.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.165+0000 7f764dbff700 1 -- 172.21.15.145:0/116578384 wait complete. 2023-08-25T04:43:11.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.166+0000 7f764dbff700 1 Processor -- start 2023-08-25T04:43:11.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.166+0000 7f764dbff700 1 -- start start 2023-08-25T04:43:11.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7648100930 0x7f76481988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76481a0440 con 0x7f76481024c0 2023-08-25T04:43:11.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76481a0580 con 0x7f7648101b20 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f764dbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76481a0710 con 0x7f7648100930 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50550/0 (socket says 172.21.15.145:50550) 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 -- 172.21.15.145:0/2701497192 learned_addr learned my addr 172.21.15.145:0/2701497192 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f76477fe700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7648100930 0x7f76481988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7646ffd700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 -- 172.21.15.145:0/2701497192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7648100930 msgr2=0x7f76481988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7648100930 0x7f76481988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 -- 172.21.15.145:0/2701497192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 msgr2=0x7f7648198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.167+0000 7f7647fff700 1 -- 172.21.15.145:0/2701497192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380095f0 con 0x7f76481024c0 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f7647fff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764000ef30 tx=0x7f764000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640018070 con 0x7f76481024c0 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764000f040 con 0x7f76481024c0 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c009520 con 0x7f76481024c0 2023-08-25T04:43:11.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000cd00 con 0x7f76481024c0 2023-08-25T04:43:11.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.168+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76481a0cc0 con 0x7f76481024c0 2023-08-25T04:43:11.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.169+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764810c190 con 0x7f76481024c0 2023-08-25T04:43:11.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.171+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f7640003a00 con 0x7f76481024c0 2023-08-25T04:43:11.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:10 smithi183 ceph-mon[131414]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:11.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3910905937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:43:11.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1759635292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:43:11.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.172+0000 7f7644ff9700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 0x7f76300a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.172+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76400caf80 con 0x7f76481024c0 2023-08-25T04:43:11.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.173+0000 7f76477fe700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 0x7f76300a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.173+0000 7f76477fe700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 0x7f76300a2410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7648101980 tx=0x7f7638009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:11.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.176+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f76400fde00 con 0x7f76481024c0 2023-08-25T04:43:11.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.393+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f764810c350 con 0x7f76481024c0 2023-08-25T04:43:11.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.393+0000 7f7644ff9700 1 -- 172.21.15.145:0/2701497192 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+600 (secure 0 0 0) 0x7f7640094bb0 con 0x7f76481024c0 2023-08-25T04:43:11.396 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/4198991683 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:11.396 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1908809275 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:11.396 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:11.397 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/227013713 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:11.397 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/417312367 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:11.397 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1961008949 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:11.397 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/3102359690 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:11.397 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T04:43:11.398 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:11.398 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T04:43:11.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 msgr2=0x7f76300a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 0x7f76300a2410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7648101980 tx=0x7f7638009310 comp rx=0 tx=0).stop 2023-08-25T04:43:11.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 msgr2=0x7f764819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764000ef30 tx=0x7f764000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 shutdown_connections 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f763009ff50 0x7f76300a2410 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76481024c0 0x7f764819fe70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7648100930 0x7f76481988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 --2- 172.21.15.145:0/2701497192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648101b20 0x7f7648198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 >> 172.21.15.145:0/2701497192 conn(0x7f76480fbea0 msgr2=0x7f76480fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 shutdown_connections 2023-08-25T04:43:11.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.396+0000 7f764dbff700 1 -- 172.21.15.145:0/2701497192 wait complete. 2023-08-25T04:43:11.401 INFO:teuthology.orchestra.run.smithi145.stderr:listed 10 entries 2023-08-25T04:43:11.415 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi145.front.sepia.ceph.com... 2023-08-25T04:43:11.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:11.416 DEBUG:teuthology.orchestra.run.smithi145:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-25T04:43:11.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi145.stderr:ceph-fuse[134890]: fuse finished with error 0 and tester_r 0 2023-08-25T04:43:11.489 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T04:43:11.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:43:11.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:43:11.536 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-25T04:43:11.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:43:11.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 -- 172.21.15.145:0/1363690036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c100930 msgr2=0x7f8f0c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 --2- 172.21.15.145:0/1363690036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c100930 0x7f8f0c100d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f080099c0 tx=0x7f8f08009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 -- 172.21.15.145:0/1363690036 shutdown_connections 2023-08-25T04:43:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 --2- 172.21.15.145:0/1363690036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 --2- 172.21.15.145:0/1363690036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c100930 0x7f8f0c100d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 --2- 172.21.15.145:0/1363690036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 -- 172.21.15.145:0/1363690036 >> 172.21.15.145:0/1363690036 conn(0x7f8f0c0fbe60 msgr2=0x7f8f0c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 -- 172.21.15.145:0/1363690036 shutdown_connections 2023-08-25T04:43:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.798+0000 7f8f13fff700 1 -- 172.21.15.145:0/1363690036 wait complete. 2023-08-25T04:43:11.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.799+0000 7f8f13fff700 1 Processor -- start 2023-08-25T04:43:11.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.799+0000 7f8f13fff700 1 -- start start 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 0x7f8f0c1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19e120 con 0x7f8f0c1024c0 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19e260 con 0x7f8f0c100930 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f13fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c19e3f0 con 0x7f8f0c101b20 2023-08-25T04:43:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f1259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f1159a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f1259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50574/0 (socket says 172.21.15.145:50574) 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f1259c700 1 -- 172.21.15.145:0/3252848931 learned_addr learned my addr 172.21.15.145:0/3252848931 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.800+0000 7f8f11d9b700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 0x7f8f0c1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 -- 172.21.15.145:0/3252848931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 msgr2=0x7f8f0c1965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 0x7f8f0c1965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 -- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 msgr2=0x7f8f0c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 -- 172.21.15.145:0/3252848931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f080095f0 con 0x7f8f0c101b20 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1259c700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f11d9b700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 0x7f8f0c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f1159a700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c196ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e9f0 tx=0x7f8efc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:11.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00c9c0 con 0x7f8f0c101b20 2023-08-25T04:43:11.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c19e6d0 con 0x7f8f0c101b20 2023-08-25T04:43:11.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8efc010430 con 0x7f8f0c101b20 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc018680 con 0x7f8f0c101b20 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.801+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c19ec30 con 0x7f8f0c101b20 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.802+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c10c140 con 0x7f8f0c101b20 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.804+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f8efc010860 con 0x7f8f0c101b20 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.805+0000 7f8f02ffd700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 0x7f8ef80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.806+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8efc014070 con 0x7f8f0c101b20 2023-08-25T04:43:11.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.806+0000 7f8f11d9b700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 0x7f8ef80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:11.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.806+0000 7f8f11d9b700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 0x7f8ef80a24e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009990 tx=0x7f8f0801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:11.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:11.808+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8efc095010 con 0x7f8f0c101b20 2023-08-25T04:43:12.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.014+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0c10c330 con 0x7f8f0c101b20 2023-08-25T04:43:12.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.017+0000 7f8f02ffd700 1 -- 172.21.15.145:0/3252848931 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f8efc098630 con 0x7f8f0c101b20 2023-08-25T04:43:12.019 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:12.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 msgr2=0x7f8ef80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 0x7f8ef80a24e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009990 tx=0x7f8f0801a040 comp rx=0 tx=0).stop 2023-08-25T04:43:12.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 msgr2=0x7f8f0c196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c196ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e9f0 tx=0x7f8efc00ed00 comp rx=0 tx=0).stop 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 shutdown_connections 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ef80a0020 0x7f8ef80a24e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c1024c0 0x7f8f0c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f0c100930 0x7f8f0c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 --2- 172.21.15.145:0/3252848931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f0c101b20 0x7f8f0c196ae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 >> 172.21.15.145:0/3252848931 conn(0x7f8f0c0fbe60 msgr2=0x7f8f0c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:12.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 shutdown_connections 2023-08-25T04:43:12.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.019+0000 7f8f13fff700 1 -- 172.21.15.145:0/3252848931 wait complete. 2023-08-25T04:43:12.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:11 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/231623774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:12.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:11 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2701497192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:12.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:43:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:11 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/231623774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:11 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2701497192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:11 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/231623774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:43:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:11 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2701497192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T04:43:12.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 -- 172.21.15.145:0/3935656370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc100920 msgr2=0x7f8ccc100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/3935656370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc100920 0x7f8ccc100d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009990 tx=0x7f8cc8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 -- 172.21.15.145:0/3935656370 shutdown_connections 2023-08-25T04:43:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/3935656370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/3935656370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc100920 0x7f8ccc100d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/3935656370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 -- 172.21.15.145:0/3935656370 >> 172.21.15.145:0/3935656370 conn(0x7f8ccc0fbe50 msgr2=0x7f8ccc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 -- 172.21.15.145:0/3935656370 shutdown_connections 2023-08-25T04:43:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.236+0000 7f8cd3bff700 1 -- 172.21.15.145:0/3935656370 wait complete. 2023-08-25T04:43:12.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 Processor -- start 2023-08-25T04:43:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 -- start start 2023-08-25T04:43:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc100920 0x7f8ccc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1a03c0 con 0x7f8ccc101b10 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1a0500 con 0x7f8ccc100920 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.237+0000 7f8cd3bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1a0690 con 0x7f8ccc1024b0 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50590/0 (socket says 172.21.15.145:50590) 2023-08-25T04:43:12.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 -- 172.21.15.145:0/107813130 learned_addr learned my addr 172.21.15.145:0/107813130 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:12.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd199b700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc100920 0x7f8ccc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd219c700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 -- 172.21.15.145:0/107813130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 msgr2=0x7f8ccc19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 -- 172.21.15.145:0/107813130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc100920 msgr2=0x7f8ccc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc100920 0x7f8ccc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 -- 172.21.15.145:0/107813130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009c40 con 0x7f8ccc101b10 2023-08-25T04:43:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd219c700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.238+0000 7f8cd119a700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00e9a0 tx=0x7f8cbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.239+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc009790 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.239+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cbc00cbd0 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.239+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80095f0 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.239+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc018680 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.239+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1a0d40 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.240+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc10c180 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.242+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f8cbc00cd40 con 0x7f8ccc101b10 2023-08-25T04:43:12.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.243+0000 7f8cc2ffd700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 0x7f8cb80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.243+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8cbc014070 con 0x7f8ccc101b10 2023-08-25T04:43:12.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.243+0000 7f8cd199b700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 0x7f8cb80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.244+0000 7f8cd199b700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 0x7f8cb80a24e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800b2e0 tx=0x7f8cc8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:12.253 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.247+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8cbc051e00 con 0x7f8ccc101b10 2023-08-25T04:43:12.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.448+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ccc10c370 con 0x7f8ccc101b10 2023-08-25T04:43:12.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.450+0000 7f8cc2ffd700 1 -- 172.21.15.145:0/107813130 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f8cbc0951e0 con 0x7f8ccc101b10 2023-08-25T04:43:12.452 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:12.452 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:12.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 msgr2=0x7f8cb80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 0x7f8cb80a24e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800b2e0 tx=0x7f8cc8005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 msgr2=0x7f8ccc198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00e9a0 tx=0x7f8cbc00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 shutdown_connections 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8cb80a0020 0x7f8cb80a24e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ccc1024b0 0x7f8ccc19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ccc100920 0x7f8ccc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 --2- 172.21.15.145:0/107813130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ccc101b10 0x7f8ccc198d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.452+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 >> 172.21.15.145:0/107813130 conn(0x7f8ccc0fbe50 msgr2=0x7f8ccc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:12.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.453+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 shutdown_connections 2023-08-25T04:43:12.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.453+0000 7f8cd3bff700 1 -- 172.21.15.145:0/107813130 wait complete. 2023-08-25T04:43:12.474 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T04:43:12.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 -- 172.21.15.145:0/2822774572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c101b00 msgr2=0x7fda2c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 --2- 172.21.15.145:0/2822774572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c101b00 0x7fda2c101f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda140098f0 tx=0x7fda14009c00 comp rx=0 tx=0).stop 2023-08-25T04:43:12.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 -- 172.21.15.145:0/2822774572 shutdown_connections 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 --2- 172.21.15.145:0/2822774572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 0x7fda2c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 --2- 172.21.15.145:0/2822774572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c100910 0x7fda2c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 --2- 172.21.15.145:0/2822774572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c101b00 0x7fda2c101f60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 -- 172.21.15.145:0/2822774572 >> 172.21.15.145:0/2822774572 conn(0x7fda2c0fbe60 msgr2=0x7fda2c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 -- 172.21.15.145:0/2822774572 shutdown_connections 2023-08-25T04:43:12.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.673+0000 7fda313ff700 1 -- 172.21.15.145:0/2822774572 wait complete. 2023-08-25T04:43:12.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.674+0000 7fda313ff700 1 Processor -- start 2023-08-25T04:43:12.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.674+0000 7fda313ff700 1 -- start start 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.674+0000 7fda313ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c100910 0x7fda2c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.674+0000 7fda313ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda313ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 0x7fda2c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda313ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0739f0 con 0x7fda2c101b00 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda313ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c073b60 con 0x7fda2c1024a0 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda313ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1a06e0 con 0x7fda2c100910 2023-08-25T04:43:12.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50608/0 (socket says 172.21.15.145:50608) 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 -- 172.21.15.145:0/2650179482 learned_addr learned my addr 172.21.15.145:0/2650179482 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda2affd700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c100910 0x7fda2c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda2b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 0x7fda2c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 -- 172.21.15.145:0/2650179482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c100910 msgr2=0x7fda2c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c100910 0x7fda2c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 -- 172.21.15.145:0/2650179482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 msgr2=0x7fda2c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 0x7fda2c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.675+0000 7fda23fff700 1 -- 172.21.15.145:0/2650179482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c0095f0 con 0x7fda2c101b00 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda23fff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda14009fd0 tx=0x7fda1400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1401c070 con 0x7fda2c101b00 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda14009520 con 0x7fda2c101b00 2023-08-25T04:43:12.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1400bc10 con 0x7fda2c101b00 2023-08-25T04:43:12.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14004a20 con 0x7fda2c101b00 2023-08-25T04:43:12.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.676+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c1a0c00 con 0x7fda2c101b00 2023-08-25T04:43:12.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.677+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c107ef0 con 0x7fda2c101b00 2023-08-25T04:43:12.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.679+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fda14020070 con 0x7fda2c101b00 2023-08-25T04:43:12.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.680+0000 7fda28ff9700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 0x7fda180a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:12.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.680+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda140cc0a0 con 0x7fda2c101b00 2023-08-25T04:43:12.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.680+0000 7fda2affd700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 0x7fda180a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:12.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.681+0000 7fda2affd700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 0x7fda180a2410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda2c101960 tx=0x7fda1c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:12.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.684+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fda1405ce00 con 0x7fda2c101b00 2023-08-25T04:43:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:12.929+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fda2c10c170 con 0x7fda2c101b00 2023-08-25T04:43:13.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:12 smithi145 ceph-mon[97681]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:13.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:12 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3252848931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:12 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/107813130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:12 smithi162 ceph-mon[130941]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:13.206 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:12 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3252848931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.206 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:12 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/107813130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.207 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:12 smithi183 ceph-mon[131414]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:13.207 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:12 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3252848931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.207 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:12 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/107813130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:13.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.774+0000 7fda28ff9700 1 -- 172.21.15.145:0/2650179482 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fda14095c70 con 0x7fda2c101b00 2023-08-25T04:43:13.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 msgr2=0x7fda180a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 0x7fda180a2410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda2c101960 tx=0x7fda1c009310 comp rx=0 tx=0).stop 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 msgr2=0x7fda2c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda14009fd0 tx=0x7fda1400b560 comp rx=0 tx=0).stop 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 shutdown_connections 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fda1809ff50 0x7fda180a2410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:13.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2c1024a0 0x7fda2c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:13.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda2c100910 0x7fda2c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:13.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 --2- 172.21.15.145:0/2650179482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2c101b00 0x7fda2c072d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:13.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 >> 172.21.15.145:0/2650179482 conn(0x7fda2c0fbe60 msgr2=0x7fda2c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:13.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 shutdown_connections 2023-08-25T04:43:13.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:13.778+0000 7fda313ff700 1 -- 172.21.15.145:0/2650179482 wait complete. 2023-08-25T04:43:13.801 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True} 2023-08-25T04:43:13.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-08-25T04:43:14.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 -- 172.21.15.145:0/3511571056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 msgr2=0x7f6578108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 --2- 172.21.15.145:0/3511571056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f6578108d70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f657000b0d0 tx=0x7f657000b3e0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 -- 172.21.15.145:0/3511571056 shutdown_connections 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 --2- 172.21.15.145:0/3511571056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f6578108d70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 --2- 172.21.15.145:0/3511571056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578100930 0x7f6578100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 --2- 172.21.15.145:0/3511571056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578101b20 0x7f6578101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 -- 172.21.15.145:0/3511571056 >> 172.21.15.145:0/3511571056 conn(0x7f65780fbea0 msgr2=0x7f65780fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 -- 172.21.15.145:0/3511571056 shutdown_connections 2023-08-25T04:43:14.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.004+0000 7f657edff700 1 -- 172.21.15.145:0/3511571056 wait complete. 2023-08-25T04:43:14.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.005+0000 7f657edff700 1 Processor -- start 2023-08-25T04:43:14.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 -- start start 2023-08-25T04:43:14.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578100930 0x7f6578198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578101b20 0x7f6578198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a0480 con 0x7f65781024c0 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a05c0 con 0x7f6578101b20 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657edff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781a0750 con 0x7f6578100930 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56552/0 (socket says 172.21.15.145:56552) 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657d39c700 1 -- 172.21.15.145:0/2308292330 learned_addr learned my addr 172.21.15.145:0/2308292330 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:14.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f657cb9b700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578100930 0x7f6578198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.006+0000 7f6577fff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578101b20 0x7f6578198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 -- 172.21.15.145:0/2308292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578100930 msgr2=0x7f6578198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578100930 0x7f6578198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 -- 172.21.15.145:0/2308292330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578101b20 msgr2=0x7f6578198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578101b20 0x7f6578198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 -- 172.21.15.145:0/2308292330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009d00 con 0x7f65781024c0 2023-08-25T04:43:14.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657d39c700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6570008ec0 tx=0x7f6570008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:14.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000e040 con 0x7f65781024c0 2023-08-25T04:43:14.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6570005070 con 0x7f65781024c0 2023-08-25T04:43:14.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65781a09d0 con 0x7f65781024c0 2023-08-25T04:43:14.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657001b6c0 con 0x7f65781024c0 2023-08-25T04:43:14.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.007+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65781a0df0 con 0x7f65781024c0 2023-08-25T04:43:14.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.008+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6578192f40 con 0x7f65781024c0 2023-08-25T04:43:14.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.010+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f65700051e0 con 0x7f65781024c0 2023-08-25T04:43:14.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.011+0000 7f6575ffb700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 0x7f65680a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.012+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f65700cc4f0 con 0x7f65781024c0 2023-08-25T04:43:14.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.012+0000 7f657cb9b700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 0x7f65680a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.012+0000 7f657cb9b700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 0x7f65680a2490 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6578101980 tx=0x7f656c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:14.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.015+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f65700960c0 con 0x7f65781024c0 2023-08-25T04:43:14.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:13 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T04:43:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:13 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T04:43:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:13 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-25T04:43:14.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.259+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f657802cc20 con 0x7f65781024c0 2023-08-25T04:43:14.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.541+0000 7f6575ffb700 1 -- 172.21.15.145:0/2308292330 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f65700996e0 con 0x7f65781024c0 2023-08-25T04:43:14.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.543+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 msgr2=0x7f65680a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.543+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 0x7f65680a2490 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6578101980 tx=0x7f656c009260 comp rx=0 tx=0).stop 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.543+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 msgr2=0x7f657819feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.543+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6570008ec0 tx=0x7f6570008ef0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 shutdown_connections 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f656809ffd0 0x7f65680a2490 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65781024c0 0x7f657819feb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578100930 0x7f6578198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 --2- 172.21.15.145:0/2308292330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6578101b20 0x7f6578198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 >> 172.21.15.145:0/2308292330 conn(0x7f65780fbea0 msgr2=0x7f65781073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:14.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 shutdown_connections 2023-08-25T04:43:14.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.544+0000 7f657edff700 1 -- 172.21.15.145:0/2308292330 wait complete. 2023-08-25T04:43:14.568 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-08-25T04:43:14.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.767+0000 7f1e027ff700 1 -- 172.21.15.145:0/2850328027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 msgr2=0x7f1dfc108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.767+0000 7f1e027ff700 1 --2- 172.21.15.145:0/2850328027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc108d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b0d0 tx=0x7f1df400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 -- 172.21.15.145:0/2850328027 shutdown_connections 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 --2- 172.21.15.145:0/2850328027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc108d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 --2- 172.21.15.145:0/2850328027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 --2- 172.21.15.145:0/2850328027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 -- 172.21.15.145:0/2850328027 >> 172.21.15.145:0/2850328027 conn(0x7f1dfc0fbea0 msgr2=0x7f1dfc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:14.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 -- 172.21.15.145:0/2850328027 shutdown_connections 2023-08-25T04:43:14.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.768+0000 7f1e027ff700 1 -- 172.21.15.145:0/2850328027 wait complete. 2023-08-25T04:43:14.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.769+0000 7f1e027ff700 1 Processor -- start 2023-08-25T04:43:14.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.769+0000 7f1e027ff700 1 -- start start 2023-08-25T04:43:14.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19e2a0 con 0x7f1dfc1024c0 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19e3e0 con 0x7f1dfc100930 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e027ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19e570 con 0x7f1dfc101b20 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e00d9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54414/0 (socket says 172.21.15.145:54414) 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfbfff700 1 -- 172.21.15.145:0/4013192829 learned_addr learned my addr 172.21.15.145:0/4013192829 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1e00d9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56560/0 (socket says 172.21.15.145:56560) 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfb7fe700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfbfff700 1 -- 172.21.15.145:0/4013192829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 msgr2=0x7f1dfc196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.770+0000 7f1dfbfff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1dfbfff700 1 -- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 msgr2=0x7f1dfc19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:14.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1dfbfff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:14.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1dfbfff700 1 -- 172.21.15.145:0/4013192829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009520 con 0x7f1dfc100930 2023-08-25T04:43:14.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1e00d9c700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:14.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1dfb7fe700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:14.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1dfbfff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ea50 tx=0x7f1dec00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:14.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00cbd0 con 0x7f1dfc100930 2023-08-25T04:43:14.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec00cd30 con 0x7f1dfc100930 2023-08-25T04:43:14.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df4009d00 con 0x7f1dfc100930 2023-08-25T04:43:14.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec018680 con 0x7f1dfc100930 2023-08-25T04:43:14.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.771+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc19ebd0 con 0x7f1dfc100930 2023-08-25T04:43:14.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.772+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc10c1a0 con 0x7f1dfc100930 2023-08-25T04:43:14.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.775+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f1dec010430 con 0x7f1dfc100930 2023-08-25T04:43:14.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.777+0000 7f1df97fa700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 0x7f1de40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:14.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.777+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dec014070 con 0x7f1dfc100930 2023-08-25T04:43:14.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.777+0000 7f1dfb7fe700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 0x7f1de40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:14.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.778+0000 7f1dfb7fe700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 0x7f1de40a2410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1df0011490 tx=0x7f1df0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:14.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:14.778+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1dec095a40 con 0x7f1dfc100930 2023-08-25T04:43:14.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:14.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T04:43:14.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:43:14.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T04:43:14.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T04:43:14.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-08-25T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:14 smithi145 ceph-mon[97681]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-08-25T04:43:15.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.026+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f1dfc107790 con 0x7f1dfc100930 2023-08-25T04:43:15.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T04:43:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-08-25T04:43:15.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:14 smithi162 ceph-mon[130941]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-08-25T04:43:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2650179482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2308292330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-08-25T04:43:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-08-25T04:43:15.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:14 smithi183 ceph-mon[131414]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-08-25T04:43:15.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.548+0000 7f1df97fa700 1 -- 172.21.15.145:0/4013192829 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1dec099060 con 0x7f1dfc100930 2023-08-25T04:43:15.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 msgr2=0x7f1de40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:15.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 0x7f1de40a2410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1df0011490 tx=0x7f1df0009260 comp rx=0 tx=0).stop 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 msgr2=0x7f1dfc196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ea50 tx=0x7f1dec00ed60 comp rx=0 tx=0).stop 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 shutdown_connections 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f1de409ff50 0x7f1de40a2410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1dfc1024c0 0x7f1dfc19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dfc100930 0x7f1dfc196720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 --2- 172.21.15.145:0/4013192829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101b20 0x7f1dfc196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 >> 172.21.15.145:0/4013192829 conn(0x7f1dfc0fbea0 msgr2=0x7f1dfc0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:15.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 shutdown_connections 2023-08-25T04:43:15.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.553+0000 7f1e027ff700 1 -- 172.21.15.145:0/4013192829 wait complete. 2023-08-25T04:43:15.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-08-25T04:43:15.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 -- 172.21.15.145:0/2227326346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 msgr2=0x7f7944105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:15.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 --2- 172.21.15.145:0/2227326346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944105f20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7938009990 tx=0x7f7938009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 -- 172.21.15.145:0/2227326346 shutdown_connections 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 --2- 172.21.15.145:0/2227326346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 0x7f79441088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 --2- 172.21.15.145:0/2227326346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f79441035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 --2- 172.21.15.145:0/2227326346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944105f20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 -- 172.21.15.145:0/2227326346 >> 172.21.15.145:0/2227326346 conn(0x7f79440fabd0 msgr2=0x7f79440fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 -- 172.21.15.145:0/2227326346 shutdown_connections 2023-08-25T04:43:15.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.784+0000 7f794359e700 1 -- 172.21.15.145:0/2227326346 wait complete. 2023-08-25T04:43:15.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.785+0000 7f794359e700 1 Processor -- start 2023-08-25T04:43:15.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.785+0000 7f794359e700 1 -- start start 2023-08-25T04:43:15.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f7944198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:15.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:15.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 0x7f794419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:15.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a0100 con 0x7f7944103b30 2023-08-25T04:43:15.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a0240 con 0x7f7944106460 2023-08-25T04:43:15.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794359e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441a03d0 con 0x7f7944101200 2023-08-25T04:43:15.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:15.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56588/0 (socket says 172.21.15.145:56588) 2023-08-25T04:43:15.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 -- 172.21.15.145:0/29160256 learned_addr learned my addr 172.21.15.145:0/29160256 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:15.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7942d9d700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 0x7f794419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:15.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794259c700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f7944198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:15.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 -- 172.21.15.145:0/29160256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 msgr2=0x7f7944198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:15.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f7944198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 -- 172.21.15.145:0/29160256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 msgr2=0x7f794419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:15.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 0x7f794419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:15.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f7941d9b700 1 -- 172.21.15.145:0/29160256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380095f0 con 0x7f7944103b30 2023-08-25T04:43:15.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.786+0000 7f794259c700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f7944198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:15.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f7941d9b700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f793800b680 tx=0x7f79380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:15.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793801d070 con 0x7f7944103b30 2023-08-25T04:43:15.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793800fbc0 con 0x7f7944103b30 2023-08-25T04:43:15.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79441a0650 con 0x7f7944103b30 2023-08-25T04:43:15.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938017650 con 0x7f7944103b30 2023-08-25T04:43:15.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.787+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79441a0a70 con 0x7f7944103b30 2023-08-25T04:43:15.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.788+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7944108060 con 0x7f7944103b30 2023-08-25T04:43:15.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.790+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f7938022470 con 0x7f7944103b30 2023-08-25T04:43:15.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.792+0000 7f79337fe700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 0x7f792c0a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:15.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.792+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f79380ccb50 con 0x7f7944103b30 2023-08-25T04:43:15.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.793+0000 7f794259c700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 0x7f792c0a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:15.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.793+0000 7f794259c700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 0x7f792c0a24b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7934009cb0 tx=0x7f7934009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:15.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:15.796+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7938096720 con 0x7f7944103b30 2023-08-25T04:43:16.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.043+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f794410bfc0 con 0x7f7944103b30 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4013192829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:active 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T04:43:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T04:43:16.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: Cluster is now healthy 2023-08-25T04:43:16.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-08-25T04:43:16.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-08-25T04:43:16.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:15 smithi145 ceph-mon[97681]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-08-25T04:43:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4013192829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:active 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T04:43:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: Cluster is now healthy 2023-08-25T04:43:16.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-08-25T04:43:16.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-08-25T04:43:16.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:15 smithi162 ceph-mon[130941]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-08-25T04:43:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4013192829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-25T04:43:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] up:active 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: Cluster is now healthy 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-08-25T04:43:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:15 smithi183 ceph-mon[131414]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-08-25T04:43:16.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.554+0000 7f79337fe700 1 -- 172.21.15.145:0/29160256 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7938099d40 con 0x7f7944103b30 2023-08-25T04:43:16.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 msgr2=0x7f792c0a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:16.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 0x7f792c0a24b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7934009cb0 tx=0x7f7934009260 comp rx=0 tx=0).stop 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 msgr2=0x7f7944198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f793800b680 tx=0x7f79380057b0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 shutdown_connections 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f792c09fff0 0x7f792c0a24b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7944106460 0x7f794419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7944101200 0x7f7944198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 --2- 172.21.15.145:0/29160256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7944103b30 0x7f7944198ac0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 >> 172.21.15.145:0/29160256 conn(0x7f79440fabd0 msgr2=0x7f79440ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:16.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 shutdown_connections 2023-08-25T04:43:16.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.558+0000 7f794359e700 1 -- 172.21.15.145:0/29160256 wait complete. 2023-08-25T04:43:16.581 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T04:43:16.582 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:43:16.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1248811712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78102b30 msgr2=0x7f4e78102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:16.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1248811712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78102b30 0x7f4e78102f50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009920 tx=0x7f4e6c009c30 comp rx=0 tx=0).stop 2023-08-25T04:43:16.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1248811712 shutdown_connections 2023-08-25T04:43:16.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1248811712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e781046c0 0x7f4e7810af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1248811712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78102b30 0x7f4e78102f50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1248811712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e78103d20 0x7f4e78104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1248811712 >> 172.21.15.145:0/1248811712 conn(0x7f4e780fe0a0 msgr2=0x7f4e781004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:16.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1248811712 shutdown_connections 2023-08-25T04:43:16.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.781+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1248811712 wait complete. 2023-08-25T04:43:16.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 Processor -- start 2023-08-25T04:43:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 -- start start 2023-08-25T04:43:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 0x7f4e78198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e781046c0 0x7f4e7819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781a03c0 con 0x7f4e78103d20 2023-08-25T04:43:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781a0500 con 0x7f4e781046c0 2023-08-25T04:43:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7f1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781a0690 con 0x7f4e78102b30 2023-08-25T04:43:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e7cf9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.782+0000 7f4e77fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 0x7f4e78198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33860/0 (socket says 172.21.15.145:33860) 2023-08-25T04:43:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 -- 172.21.15.145:0/1064036058 learned_addr learned my addr 172.21.15.145:0/1064036058 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7d79c700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e781046c0 0x7f4e7819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 -- 172.21.15.145:0/1064036058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e781046c0 msgr2=0x7f4e7819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:16.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e781046c0 0x7f4e7819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 -- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 msgr2=0x7f4e78198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:16.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 0x7f4e78198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 -- 172.21.15.145:0/1064036058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640095f0 con 0x7f4e78102b30 2023-08-25T04:43:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e77fff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 0x7f4e78198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7cf9b700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009230 tx=0x7f4e6c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c01c070 con 0x7f4e78102b30 2023-08-25T04:43:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6c00f7d0 con 0x7f4e78102b30 2023-08-25T04:43:16.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c009520 con 0x7f4e78102b30 2023-08-25T04:43:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.783+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c004930 con 0x7f4e78102b30 2023-08-25T04:43:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.784+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781a0b50 con 0x7f4e78102b30 2023-08-25T04:43:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.785+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7810a160 con 0x7f4e78102b30 2023-08-25T04:43:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.786+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4e6c020070 con 0x7f4e78102b30 2023-08-25T04:43:16.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.788+0000 7f4e75ffb700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 0x7f4e680a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:16.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.788+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e6c0cc0e0 con 0x7f4e78102b30 2023-08-25T04:43:16.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.788+0000 7f4e77fff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 0x7f4e680a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:16.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.788+0000 7f4e77fff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 0x7f4e680a2460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e64005fd0 tx=0x7f4e64009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:16.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:16.790+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4e6c095cb0 con 0x7f4e78102b30 2023-08-25T04:43:16.828 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: 2023-08-25T04:43:16.547+0000 7f558a4a8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:active 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-08-25T04:43:16.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:16 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:43:17.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.034+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e7810e5e0 con 0x7f4e78102b30 2023-08-25T04:43:17.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.036+0000 7f4e75ffb700 1 -- 172.21.15.145:0/1064036058 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f4e6c0992d0 con 0x7f4e78102b30 2023-08-25T04:43:17.038 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:17.038 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:43:17.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 msgr2=0x7f4e680a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 0x7f4e680a2460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e64005fd0 tx=0x7f4e64009310 comp rx=0 tx=0).stop 2023-08-25T04:43:17.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 msgr2=0x7f4e78198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009230 tx=0x7f4e6c00b790 comp rx=0 tx=0).stop 2023-08-25T04:43:17.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 shutdown_connections 2023-08-25T04:43:17.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4e6809ffa0 0x7f4e680a2460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e781046c0 0x7f4e7819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e78102b30 0x7f4e78198840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 --2- 172.21.15.145:0/1064036058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e78103d20 0x7f4e78198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 >> 172.21.15.145:0/1064036058 conn(0x7f4e780fe0a0 msgr2=0x7f4e781072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:17.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 shutdown_connections 2023-08-25T04:43:17.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.038+0000 7f4e7f1ff700 1 -- 172.21.15.145:0/1064036058 wait complete. 2023-08-25T04:43:17.045 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:43:17.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:43:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-25T04:43:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T04:43:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:active 2023-08-25T04:43:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T04:43:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-08-25T04:43:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:16 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:43:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-08-25T04:43:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-25T04:43:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] up:active 2023-08-25T04:43:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/29160256' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-25T04:43:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-08-25T04:43:17.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:16 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:43:17.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.256+0000 7f30eabff700 1 -- 172.21.15.145:0/2383579400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e4101b20 msgr2=0x7f30e4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.256+0000 7f30eabff700 1 --2- 172.21.15.145:0/2383579400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e4101b20 0x7f30e4101f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009920 tx=0x7f30d0009c30 comp rx=0 tx=0).stop 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 -- 172.21.15.145:0/2383579400 shutdown_connections 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 --2- 172.21.15.145:0/2383579400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e41024c0 0x7f30e4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 --2- 172.21.15.145:0/2383579400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 --2- 172.21.15.145:0/2383579400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e4101b20 0x7f30e4101f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 -- 172.21.15.145:0/2383579400 >> 172.21.15.145:0/2383579400 conn(0x7f30e40fbe60 msgr2=0x7f30e40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:17.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 -- 172.21.15.145:0/2383579400 shutdown_connections 2023-08-25T04:43:17.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.257+0000 7f30eabff700 1 -- 172.21.15.145:0/2383579400 wait complete. 2023-08-25T04:43:17.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 Processor -- start 2023-08-25T04:43:17.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 -- start start 2023-08-25T04:43:17.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 0x7f30e4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e41024c0 0x7f30e419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a0430 con 0x7f30e4100930 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a0570 con 0x7f30e41024c0 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30eabff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41a0700 con 0x7f30e4101b20 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30e899b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30e899b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56624/0 (socket says 172.21.15.145:56624) 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30e899b700 1 -- 172.21.15.145:0/483627766 learned_addr learned my addr 172.21.15.145:0/483627766 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.258+0000 7f30e3fff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 0x7f30e4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e919c700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e41024c0 0x7f30e419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 -- 172.21.15.145:0/483627766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 msgr2=0x7f30e4198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 0x7f30e4198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 -- 172.21.15.145:0/483627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e41024c0 msgr2=0x7f30e419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e41024c0 0x7f30e419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 -- 172.21.15.145:0/483627766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc009d00 con 0x7f30e4100930 2023-08-25T04:43:17.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e3fff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 0x7f30e4198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:17.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e899b700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30d800e970 tx=0x7f30d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:17.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8009790 con 0x7f30e4100930 2023-08-25T04:43:17.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30d800c7e0 con 0x7f30e4100930 2023-08-25T04:43:17.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8003c70 con 0x7f30e4100930 2023-08-25T04:43:17.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d0009520 con 0x7f30e4100930 2023-08-25T04:43:17.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.259+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e41a0d10 con 0x7f30e4100930 2023-08-25T04:43:17.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.261+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e410c190 con 0x7f30e4100930 2023-08-25T04:43:17.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.262+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f30d800cc50 con 0x7f30e4100930 2023-08-25T04:43:17.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.265+0000 7f30e1ffb700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 0x7f30d40a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.265+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f30d8014070 con 0x7f30e4100930 2023-08-25T04:43:17.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.265+0000 7f30e3fff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 0x7f30d40a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.266+0000 7f30e3fff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 0x7f30d40a2460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f30d000fa40 tx=0x7f30d0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:17.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.269+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f30d8095150 con 0x7f30e4100930 2023-08-25T04:43:17.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.471+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30e410c3d0 con 0x7f30e4100930 2023-08-25T04:43:17.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.473+0000 7f30e1ffb700 1 -- 172.21.15.145:0/483627766 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f30d8003dd0 con 0x7f30e4100930 2023-08-25T04:43:17.475 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:17.475 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:17.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.475+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 msgr2=0x7f30d40a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.475+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 0x7f30d40a2460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f30d000fa40 tx=0x7f30d0019040 comp rx=0 tx=0).stop 2023-08-25T04:43:17.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.475+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 msgr2=0x7f30e41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.475+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30d800e970 tx=0x7f30d800ec80 comp rx=0 tx=0).stop 2023-08-25T04:43:17.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 shutdown_connections 2023-08-25T04:43:17.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f30d409ffa0 0x7f30d40a2460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30e41024c0 0x7f30e419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e4100930 0x7f30e41988b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 --2- 172.21.15.145:0/483627766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4101b20 0x7f30e4198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 >> 172.21.15.145:0/483627766 conn(0x7f30e40fbe60 msgr2=0x7f30e40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:17.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 shutdown_connections 2023-08-25T04:43:17.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.476+0000 7f30eabff700 1 -- 172.21.15.145:0/483627766 wait complete. 2023-08-25T04:43:17.498 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T04:43:17.499 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T04:43:17.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 -- 172.21.15.145:0/3031466809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368100930 msgr2=0x7fc368100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 --2- 172.21.15.145:0/3031466809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368100930 0x7fc368100d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0099c0 tx=0x7fc35c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 -- 172.21.15.145:0/3031466809 shutdown_connections 2023-08-25T04:43:17.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 --2- 172.21.15.145:0/3031466809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3681024c0 0x7fc368108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 --2- 172.21.15.145:0/3031466809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368100930 0x7fc368100d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 --2- 172.21.15.145:0/3031466809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc368101b20 0x7fc368101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 -- 172.21.15.145:0/3031466809 >> 172.21.15.145:0/3031466809 conn(0x7fc3680fbe60 msgr2=0x7fc3680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 -- 172.21.15.145:0/3031466809 shutdown_connections 2023-08-25T04:43:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.695+0000 7fc36efff700 1 -- 172.21.15.145:0/3031466809 wait complete. 2023-08-25T04:43:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.696+0000 7fc36efff700 1 Processor -- start 2023-08-25T04:43:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.696+0000 7fc36efff700 1 -- start start 2023-08-25T04:43:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368101b20 0x7fc368072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3681024c0 0x7fc368073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680739f0 con 0x7fc368100930 2023-08-25T04:43:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368073b60 con 0x7fc368101b20 2023-08-25T04:43:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36efff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681a0650 con 0x7fc3681024c0 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56652/0 (socket says 172.21.15.145:56652) 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 -- 172.21.15.145:0/1393195049 learned_addr learned my addr 172.21.15.145:0/1393195049 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc367fff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368101b20 0x7fc368072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36d59c700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3681024c0 0x7fc368073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 -- 172.21.15.145:0/1393195049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3681024c0 msgr2=0x7fc368073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3681024c0 0x7fc368073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 -- 172.21.15.145:0/1393195049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368101b20 msgr2=0x7fc368072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368101b20 0x7fc368072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.697+0000 7fc36cd9b700 1 -- 172.21.15.145:0/1393195049 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0095f0 con 0x7fc368100930 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc36cd9b700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009990 tx=0x7fc35c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c01d070 con 0x7fc368100930 2023-08-25T04:43:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c0054f0 con 0x7fc368100930 2023-08-25T04:43:17.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004b00 con 0x7fc368100930 2023-08-25T04:43:17.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3681a08d0 con 0x7fc368100930 2023-08-25T04:43:17.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.698+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681a0cf0 con 0x7fc368100930 2023-08-25T04:43:17.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.700+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36810c190 con 0x7fc368100930 2023-08-25T04:43:17.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.701+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc35c00fb80 con 0x7fc368100930 2023-08-25T04:43:17.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.703+0000 7fc365ffb700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 0x7fc3580a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:17.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.703+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc35c0cc190 con 0x7fc368100930 2023-08-25T04:43:17.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.703+0000 7fc367fff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 0x7fc3580a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:17.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.703+0000 7fc367fff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 0x7fc3580a24e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc354009c60 tx=0x7fc354009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:17.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.706+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc35c095d60 con 0x7fc368100930 2023-08-25T04:43:17.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.911+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc368060d60 con 0x7fc3580a0020 2023-08-25T04:43:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.926+0000 7fc365ffb700 1 -- 172.21.15.145:0/1393195049 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422435 (secure 0 0 0) 0x7fc368060d60 con 0x7fc3580a0020 2023-08-25T04:43:17.935 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:17.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 msgr2=0x7fc3580a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 0x7fc3580a24e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc354009c60 tx=0x7fc354009260 comp rx=0 tx=0).stop 2023-08-25T04:43:17.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 msgr2=0x7fc3680746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009990 tx=0x7fc35c00f690 comp rx=0 tx=0).stop 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 shutdown_connections 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc3580a0020 0x7fc3580a24e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3681024c0 0x7fc368073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc368100930 0x7fc3680746d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 --2- 172.21.15.145:0/1393195049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc368101b20 0x7fc368072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:17.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 >> 172.21.15.145:0/1393195049 conn(0x7fc3680fbe60 msgr2=0x7fc3680fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:17.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 shutdown_connections 2023-08-25T04:43:17.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:17.931+0000 7fc36efff700 1 -- 172.21.15.145:0/1393195049 wait complete. 2023-08-25T04:43:17.940 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:43:17.953 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-08-25T04:43:17.689930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":230,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":180,"ondisk_log_size":180,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335492,"kb_used_data":11244,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606332,"statfs":{"total":1151940427776,"available":1151596883968,"internally_reserved":0,"allocated":11513856,"data_stored":7350537,"data_compressed":247383,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003390"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498548+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:42:55.498548+0000","last_peered":"2023-08-25T04:42:55.498548+0000","last_clean":"2023-08-25T04:42:55.498548+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:42:55.498548+0000","last_undegraded":"2023-08-25T04:42:55.498548+0000","last_fullsized":"2023-08-25T04:42:55.498548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:11.466599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498351+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:42:55.498351+0000","last_peered":"2023-08-25T04:42:55.498351+0000","last_clean":"2023-08-25T04:42:55.498351+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:42:55.498351+0000","last_undegraded":"2023-08-25T04:42:55.498351+0000","last_fullsized":"2023-08-25T04:42:55.498351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:40.571648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498628+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:42:55.498628+0000","last_peered":"2023-08-25T04:42:55.498628+0000","last_clean":"2023-08-25T04:42:55.498628+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:42:55.498628+0000","last_undegraded":"2023-08-25T04:42:55.498628+0000","last_fullsized":"2023-08-25T04:42:55.498628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:23.547350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498876+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:42:55.498876+0000","last_peered":"2023-08-25T04:42:55.498876+0000","last_clean":"2023-08-25T04:42:55.498876+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:42:55.498876+0000","last_undegraded":"2023-08-25T04:42:55.498876+0000","last_fullsized":"2023-08-25T04:42:55.498876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:12.102282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331964+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:42:55.331964+0000","last_peered":"2023-08-25T04:42:55.331964+0000","last_clean":"2023-08-25T04:42:55.331964+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:42:55.331964+0000","last_undegraded":"2023-08-25T04:42:55.331964+0000","last_fullsized":"2023-08-25T04:42:55.331964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:11.148176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499676+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:42:55.499676+0000","last_peered":"2023-08-25T04:42:55.499676+0000","last_clean":"2023-08-25T04:42:55.499676+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:42:55.499676+0000","last_undegraded":"2023-08-25T04:42:55.499676+0000","last_fullsized":"2023-08-25T04:42:55.499676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:49.556742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332092+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:42:55.332092+0000","last_peered":"2023-08-25T04:42:55.332092+0000","last_clean":"2023-08-25T04:42:55.332092+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:42:55.332092+0000","last_undegraded":"2023-08-25T04:42:55.332092+0000","last_fullsized":"2023-08-25T04:42:55.332092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:02.949033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497903+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:42:55.497903+0000","last_peered":"2023-08-25T04:42:55.497903+0000","last_clean":"2023-08-25T04:42:55.497903+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:42:55.497903+0000","last_undegraded":"2023-08-25T04:42:55.497903+0000","last_fullsized":"2023-08-25T04:42:55.497903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:51:08.670149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498711+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:42:55.498711+0000","last_peered":"2023-08-25T04:42:55.498711+0000","last_clean":"2023-08-25T04:42:55.498711+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:42:55.498711+0000","last_undegraded":"2023-08-25T04:42:55.498711+0000","last_fullsized":"2023-08-25T04:42:55.498711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:50:22.640028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499835+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:42:55.499835+0000","last_peered":"2023-08-25T04:42:55.499835+0000","last_clean":"2023-08-25T04:42:55.499835+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:42:55.499835+0000","last_undegraded":"2023-08-25T04:42:55.499835+0000","last_fullsized":"2023-08-25T04:42:55.499835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:12:37.196016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332636+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:42:55.332636+0000","last_peered":"2023-08-25T04:42:55.332636+0000","last_clean":"2023-08-25T04:42:55.332636+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:42:55.332636+0000","last_undegraded":"2023-08-25T04:42:55.332636+0000","last_fullsized":"2023-08-25T04:42:55.332636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:30.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498731+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:42:55.498731+0000","last_peered":"2023-08-25T04:42:55.498731+0000","last_clean":"2023-08-25T04:42:55.498731+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:42:55.498731+0000","last_undegraded":"2023-08-25T04:42:55.498731+0000","last_fullsized":"2023-08-25T04:42:55.498731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:42.786018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038549+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:42:56.038549+0000","last_peered":"2023-08-25T04:42:56.038549+0000","last_clean":"2023-08-25T04:42:56.038549+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:42:56.038549+0000","last_undegraded":"2023-08-25T04:42:56.038549+0000","last_fullsized":"2023-08-25T04:42:56.038549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:16:22.636180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498103+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:42:55.498103+0000","last_peered":"2023-08-25T04:42:55.498103+0000","last_clean":"2023-08-25T04:42:55.498103+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:42:55.498103+0000","last_undegraded":"2023-08-25T04:42:55.498103+0000","last_fullsized":"2023-08-25T04:42:55.498103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:04.859467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498197+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:42:55.498197+0000","last_peered":"2023-08-25T04:42:55.498197+0000","last_clean":"2023-08-25T04:42:55.498197+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:42:55.498197+0000","last_undegraded":"2023-08-25T04:42:55.498197+0000","last_fullsized":"2023-08-25T04:42:55.498197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:08.029332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499673+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:42:55.499673+0000","last_peered":"2023-08-25T04:42:55.499673+0000","last_clean":"2023-08-25T04:42:55.499673+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:42:55.499673+0000","last_undegraded":"2023-08-25T04:42:55.499673+0000","last_fullsized":"2023-08-25T04:42:55.499673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:03.011607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498394+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:42:55.498394+0000","last_peered":"2023-08-25T04:42:55.498394+0000","last_clean":"2023-08-25T04:42:55.498394+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:42:55.498394+0000","last_undegraded":"2023-08-25T04:42:55.498394+0000","last_fullsized":"2023-08-25T04:42:55.498394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:09.299405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331350+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:42:55.331350+0000","last_peered":"2023-08-25T04:42:55.331350+0000","last_clean":"2023-08-25T04:42:55.331350+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:42:55.331350+0000","last_undegraded":"2023-08-25T04:42:55.331350+0000","last_fullsized":"2023-08-25T04:42:55.331350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:47.697095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499136+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:42:55.499136+0000","last_peered":"2023-08-25T04:42:55.499136+0000","last_clean":"2023-08-25T04:42:55.499136+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:42:55.499136+0000","last_undegraded":"2023-08-25T04:42:55.499136+0000","last_fullsized":"2023-08-25T04:42:55.499136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:03.430027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499400+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:42:55.499400+0000","last_peered":"2023-08-25T04:42:55.499400+0000","last_clean":"2023-08-25T04:42:55.499400+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:42:55.499400+0000","last_undegraded":"2023-08-25T04:42:55.499400+0000","last_fullsized":"2023-08-25T04:42:55.499400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:26.766631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331429+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:42:55.331429+0000","last_peered":"2023-08-25T04:42:55.331429+0000","last_clean":"2023-08-25T04:42:55.331429+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:42:55.331429+0000","last_undegraded":"2023-08-25T04:42:55.331429+0000","last_fullsized":"2023-08-25T04:42:55.331429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:49:03.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498335+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:42:55.498335+0000","last_peered":"2023-08-25T04:42:55.498335+0000","last_clean":"2023-08-25T04:42:55.498335+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:42:55.498335+0000","last_undegraded":"2023-08-25T04:42:55.498335+0000","last_fullsized":"2023-08-25T04:42:55.498335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:38:30.988002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498343+0000","last_change":"2023-08-25T04:42:54.362609+0000","last_active":"2023-08-25T04:42:55.498343+0000","last_peered":"2023-08-25T04:42:55.498343+0000","last_clean":"2023-08-25T04:42:55.498343+0000","last_became_active":"2023-08-25T04:42:54.362224+0000","last_became_peered":"2023-08-25T04:42:54.362224+0000","last_unstale":"2023-08-25T04:42:55.498343+0000","last_undegraded":"2023-08-25T04:42:55.498343+0000","last_fullsized":"2023-08-25T04:42:55.498343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:21:07.957008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498561+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:42:55.498561+0000","last_peered":"2023-08-25T04:42:55.498561+0000","last_clean":"2023-08-25T04:42:55.498561+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:42:55.498561+0000","last_undegraded":"2023-08-25T04:42:55.498561+0000","last_fullsized":"2023-08-25T04:42:55.498561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:04.868337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498291+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:42:55.498291+0000","last_peered":"2023-08-25T04:42:55.498291+0000","last_clean":"2023-08-25T04:42:55.498291+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:42:55.498291+0000","last_undegraded":"2023-08-25T04:42:55.498291+0000","last_fullsized":"2023-08-25T04:42:55.498291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:50.565716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499241+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:42:55.499241+0000","last_peered":"2023-08-25T04:42:55.499241+0000","last_clean":"2023-08-25T04:42:55.499241+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:42:55.499241+0000","last_undegraded":"2023-08-25T04:42:55.499241+0000","last_fullsized":"2023-08-25T04:42:55.499241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:14.665503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332300+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:42:55.332300+0000","last_peered":"2023-08-25T04:42:55.332300+0000","last_clean":"2023-08-25T04:42:55.332300+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:42:55.332300+0000","last_undegraded":"2023-08-25T04:42:55.332300+0000","last_fullsized":"2023-08-25T04:42:55.332300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:00:31.012805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498445+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:42:55.498445+0000","last_peered":"2023-08-25T04:42:55.498445+0000","last_clean":"2023-08-25T04:42:55.498445+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:42:55.498445+0000","last_undegraded":"2023-08-25T04:42:55.498445+0000","last_fullsized":"2023-08-25T04:42:55.498445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:20:46.135430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498850+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:42:55.498850+0000","last_peered":"2023-08-25T04:42:55.498850+0000","last_clean":"2023-08-25T04:42:55.498850+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:42:55.498850+0000","last_undegraded":"2023-08-25T04:42:55.498850+0000","last_fullsized":"2023-08-25T04:42:55.498850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:22:03.190298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332210+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:42:55.332210+0000","last_peered":"2023-08-25T04:42:55.332210+0000","last_clean":"2023-08-25T04:42:55.332210+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:42:55.332210+0000","last_undegraded":"2023-08-25T04:42:55.332210+0000","last_fullsized":"2023-08-25T04:42:55.332210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:10.908437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500392+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:42:55.500392+0000","last_peered":"2023-08-25T04:42:55.500392+0000","last_clean":"2023-08-25T04:42:55.500392+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:42:55.500392+0000","last_undegraded":"2023-08-25T04:42:55.500392+0000","last_fullsized":"2023-08-25T04:42:55.500392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:13:17.680357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498603+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:42:55.498603+0000","last_peered":"2023-08-25T04:42:55.498603+0000","last_clean":"2023-08-25T04:42:55.498603+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:42:55.498603+0000","last_undegraded":"2023-08-25T04:42:55.498603+0000","last_fullsized":"2023-08-25T04:42:55.498603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:29.499407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498941+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:42:55.498941+0000","last_peered":"2023-08-25T04:42:55.498941+0000","last_clean":"2023-08-25T04:42:55.498941+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:42:55.498941+0000","last_undegraded":"2023-08-25T04:42:55.498941+0000","last_fullsized":"2023-08-25T04:42:55.498941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:45.183593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332275+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:42:55.332275+0000","last_peered":"2023-08-25T04:42:55.332275+0000","last_clean":"2023-08-25T04:42:55.332275+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:42:55.332275+0000","last_undegraded":"2023-08-25T04:42:55.332275+0000","last_fullsized":"2023-08-25T04:42:55.332275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:39.960781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498543+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:42:55.498543+0000","last_peered":"2023-08-25T04:42:55.498543+0000","last_clean":"2023-08-25T04:42:55.498543+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:42:55.498543+0000","last_undegraded":"2023-08-25T04:42:55.498543+0000","last_fullsized":"2023-08-25T04:42:55.498543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:58:01.500963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499498+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:42:55.499498+0000","last_peered":"2023-08-25T04:42:55.499498+0000","last_clean":"2023-08-25T04:42:55.499498+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:42:55.499498+0000","last_undegraded":"2023-08-25T04:42:55.499498+0000","last_fullsized":"2023-08-25T04:42:55.499498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:48.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499063+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:42:55.499063+0000","last_peered":"2023-08-25T04:42:55.499063+0000","last_clean":"2023-08-25T04:42:55.499063+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:42:55.499063+0000","last_undegraded":"2023-08-25T04:42:55.499063+0000","last_fullsized":"2023-08-25T04:42:55.499063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:26.652793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498234+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:42:55.498234+0000","last_peered":"2023-08-25T04:42:55.498234+0000","last_clean":"2023-08-25T04:42:55.498234+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:42:55.498234+0000","last_undegraded":"2023-08-25T04:42:55.498234+0000","last_fullsized":"2023-08-25T04:42:55.498234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:18.199795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038361+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:42:56.038361+0000","last_peered":"2023-08-25T04:42:56.038361+0000","last_clean":"2023-08-25T04:42:56.038361+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:42:56.038361+0000","last_undegraded":"2023-08-25T04:42:56.038361+0000","last_fullsized":"2023-08-25T04:42:56.038361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:28.752707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497778+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:42:55.497778+0000","last_peered":"2023-08-25T04:42:55.497778+0000","last_clean":"2023-08-25T04:42:55.497778+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:42:55.497778+0000","last_undegraded":"2023-08-25T04:42:55.497778+0000","last_fullsized":"2023-08-25T04:42:55.497778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:35:07.264982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498249+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:42:55.498249+0000","last_peered":"2023-08-25T04:42:55.498249+0000","last_clean":"2023-08-25T04:42:55.498249+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:42:55.498249+0000","last_undegraded":"2023-08-25T04:42:55.498249+0000","last_fullsized":"2023-08-25T04:42:55.498249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:12.166934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498231+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:42:55.498231+0000","last_peered":"2023-08-25T04:42:55.498231+0000","last_clean":"2023-08-25T04:42:55.498231+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:42:55.498231+0000","last_undegraded":"2023-08-25T04:42:55.498231+0000","last_fullsized":"2023-08-25T04:42:55.498231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:51.295038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330846+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:42:55.330846+0000","last_peered":"2023-08-25T04:42:55.330846+0000","last_clean":"2023-08-25T04:42:55.330846+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:42:55.330846+0000","last_undegraded":"2023-08-25T04:42:55.330846+0000","last_fullsized":"2023-08-25T04:42:55.330846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:47.527846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497962+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:42:55.497962+0000","last_peered":"2023-08-25T04:42:55.497962+0000","last_clean":"2023-08-25T04:42:55.497962+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:42:55.497962+0000","last_undegraded":"2023-08-25T04:42:55.497962+0000","last_fullsized":"2023-08-25T04:42:55.497962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:56:30.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499596+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:42:55.499596+0000","last_peered":"2023-08-25T04:42:55.499596+0000","last_clean":"2023-08-25T04:42:55.499596+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:42:55.499596+0000","last_undegraded":"2023-08-25T04:42:55.499596+0000","last_fullsized":"2023-08-25T04:42:55.499596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:44.399055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498401+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:42:55.498401+0000","last_peered":"2023-08-25T04:42:55.498401+0000","last_clean":"2023-08-25T04:42:55.498401+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:42:55.498401+0000","last_undegraded":"2023-08-25T04:42:55.498401+0000","last_fullsized":"2023-08-25T04:42:55.498401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:05:40.395184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330846+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:42:55.330846+0000","last_peered":"2023-08-25T04:42:55.330846+0000","last_clean":"2023-08-25T04:42:55.330846+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:42:55.330846+0000","last_undegraded":"2023-08-25T04:42:55.330846+0000","last_fullsized":"2023-08-25T04:42:55.330846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:19:16.538915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332437+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:42:55.332437+0000","last_peered":"2023-08-25T04:42:55.332437+0000","last_clean":"2023-08-25T04:42:55.332437+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:42:55.332437+0000","last_undegraded":"2023-08-25T04:42:55.332437+0000","last_fullsized":"2023-08-25T04:42:55.332437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:49:29.145032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498462+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:42:55.498462+0000","last_peered":"2023-08-25T04:42:55.498462+0000","last_clean":"2023-08-25T04:42:55.498462+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:42:55.498462+0000","last_undegraded":"2023-08-25T04:42:55.498462+0000","last_fullsized":"2023-08-25T04:42:55.498462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:22.343180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499835+0000","last_change":"2023-08-25T04:42:54.357939+0000","last_active":"2023-08-25T04:42:55.499835+0000","last_peered":"2023-08-25T04:42:55.499835+0000","last_clean":"2023-08-25T04:42:55.499835+0000","last_became_active":"2023-08-25T04:42:54.357449+0000","last_became_peered":"2023-08-25T04:42:54.357449+0000","last_unstale":"2023-08-25T04:42:55.499835+0000","last_undegraded":"2023-08-25T04:42:55.499835+0000","last_fullsized":"2023-08-25T04:42:55.499835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:20:54.583324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500361+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:42:55.500361+0000","last_peered":"2023-08-25T04:42:55.500361+0000","last_clean":"2023-08-25T04:42:55.500361+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:42:55.500361+0000","last_undegraded":"2023-08-25T04:42:55.500361+0000","last_fullsized":"2023-08-25T04:42:55.500361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:17.153473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498682+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:42:55.498682+0000","last_peered":"2023-08-25T04:42:55.498682+0000","last_clean":"2023-08-25T04:42:55.498682+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:42:55.498682+0000","last_undegraded":"2023-08-25T04:42:55.498682+0000","last_fullsized":"2023-08-25T04:42:55.498682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:30:28.593283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330967+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:42:55.330967+0000","last_peered":"2023-08-25T04:42:55.330967+0000","last_clean":"2023-08-25T04:42:55.330967+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:42:55.330967+0000","last_undegraded":"2023-08-25T04:42:55.330967+0000","last_fullsized":"2023-08-25T04:42:55.330967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:35.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:43:05.979386+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:43:05.979386+0000","last_peered":"2023-08-25T04:43:05.979386+0000","last_clean":"2023-08-25T04:43:05.979386+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:43:05.979386+0000","last_undegraded":"2023-08-25T04:43:05.979386+0000","last_fullsized":"2023-08-25T04:43:05.979386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:14.984411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332094+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:42:55.332094+0000","last_peered":"2023-08-25T04:42:55.332094+0000","last_clean":"2023-08-25T04:42:55.332094+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:42:55.332094+0000","last_undegraded":"2023-08-25T04:42:55.332094+0000","last_fullsized":"2023-08-25T04:42:55.332094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:17:28.690442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500617+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:42:55.500617+0000","last_peered":"2023-08-25T04:42:55.500617+0000","last_clean":"2023-08-25T04:42:55.500617+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:42:55.500617+0000","last_undegraded":"2023-08-25T04:42:55.500617+0000","last_fullsized":"2023-08-25T04:42:55.500617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:15.202392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498872+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:42:55.498872+0000","last_peered":"2023-08-25T04:42:55.498872+0000","last_clean":"2023-08-25T04:42:55.498872+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:42:55.498872+0000","last_undegraded":"2023-08-25T04:42:55.498872+0000","last_fullsized":"2023-08-25T04:42:55.498872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:58.673573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038450+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:42:56.038450+0000","last_peered":"2023-08-25T04:42:56.038450+0000","last_clean":"2023-08-25T04:42:56.038450+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:42:56.038450+0000","last_undegraded":"2023-08-25T04:42:56.038450+0000","last_fullsized":"2023-08-25T04:42:56.038450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:37.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498542+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:42:55.498542+0000","last_peered":"2023-08-25T04:42:55.498542+0000","last_clean":"2023-08-25T04:42:55.498542+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:42:55.498542+0000","last_undegraded":"2023-08-25T04:42:55.498542+0000","last_fullsized":"2023-08-25T04:42:55.498542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:31.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498649+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:42:55.498649+0000","last_peered":"2023-08-25T04:42:55.498649+0000","last_clean":"2023-08-25T04:42:55.498649+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:42:55.498649+0000","last_undegraded":"2023-08-25T04:42:55.498649+0000","last_fullsized":"2023-08-25T04:42:55.498649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:20.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332674+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:42:55.332674+0000","last_peered":"2023-08-25T04:42:55.332674+0000","last_clean":"2023-08-25T04:42:55.332674+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:42:55.332674+0000","last_undegraded":"2023-08-25T04:42:55.332674+0000","last_fullsized":"2023-08-25T04:42:55.332674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:49.143880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331296+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:42:55.331296+0000","last_peered":"2023-08-25T04:42:55.331296+0000","last_clean":"2023-08-25T04:42:55.331296+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:42:55.331296+0000","last_undegraded":"2023-08-25T04:42:55.331296+0000","last_fullsized":"2023-08-25T04:42:55.331296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:00:30.917396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499926+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:42:55.499926+0000","last_peered":"2023-08-25T04:42:55.499926+0000","last_clean":"2023-08-25T04:42:55.499926+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:42:55.499926+0000","last_undegraded":"2023-08-25T04:42:55.499926+0000","last_fullsized":"2023-08-25T04:42:55.499926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:59:16.962246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332401+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:42:55.332401+0000","last_peered":"2023-08-25T04:42:55.332401+0000","last_clean":"2023-08-25T04:42:55.332401+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:42:55.332401+0000","last_undegraded":"2023-08-25T04:42:55.332401+0000","last_fullsized":"2023-08-25T04:42:55.332401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:13:40.041773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038956+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:42:56.038956+0000","last_peered":"2023-08-25T04:42:56.038956+0000","last_clean":"2023-08-25T04:42:56.038956+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:42:56.038956+0000","last_undegraded":"2023-08-25T04:42:56.038956+0000","last_fullsized":"2023-08-25T04:42:56.038956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:25.874788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331742+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:42:55.331742+0000","last_peered":"2023-08-25T04:42:55.331742+0000","last_clean":"2023-08-25T04:42:55.331742+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:42:55.331742+0000","last_undegraded":"2023-08-25T04:42:55.331742+0000","last_fullsized":"2023-08-25T04:42:55.331742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:05:47.949354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500166+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:42:55.500166+0000","last_peered":"2023-08-25T04:42:55.500166+0000","last_clean":"2023-08-25T04:42:55.500166+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:42:55.500166+0000","last_undegraded":"2023-08-25T04:42:55.500166+0000","last_fullsized":"2023-08-25T04:42:55.500166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:09:57.787096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500037+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:42:55.500037+0000","last_peered":"2023-08-25T04:42:55.500037+0000","last_clean":"2023-08-25T04:42:55.500037+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:42:55.500037+0000","last_undegraded":"2023-08-25T04:42:55.500037+0000","last_fullsized":"2023-08-25T04:42:55.500037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:43:06.007419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332208+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:42:55.332208+0000","last_peered":"2023-08-25T04:42:55.332208+0000","last_clean":"2023-08-25T04:42:55.332208+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:42:55.332208+0000","last_undegraded":"2023-08-25T04:42:55.332208+0000","last_fullsized":"2023-08-25T04:42:55.332208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:20:47.744053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498670+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:42:55.498670+0000","last_peered":"2023-08-25T04:42:55.498670+0000","last_clean":"2023-08-25T04:42:55.498670+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:42:55.498670+0000","last_undegraded":"2023-08-25T04:42:55.498670+0000","last_fullsized":"2023-08-25T04:42:55.498670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:37.815247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331991+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:42:55.331991+0000","last_peered":"2023-08-25T04:42:55.331991+0000","last_clean":"2023-08-25T04:42:55.331991+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:42:55.331991+0000","last_undegraded":"2023-08-25T04:42:55.331991+0000","last_fullsized":"2023-08-25T04:42:55.331991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:13.490903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331139+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:42:55.331139+0000","last_peered":"2023-08-25T04:42:55.331139+0000","last_clean":"2023-08-25T04:42:55.331139+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:42:55.331139+0000","last_undegraded":"2023-08-25T04:42:55.331139+0000","last_fullsized":"2023-08-25T04:42:55.331139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:01.814559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499322+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:42:55.499322+0000","last_peered":"2023-08-25T04:42:55.499322+0000","last_clean":"2023-08-25T04:42:55.499322+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:42:55.499322+0000","last_undegraded":"2023-08-25T04:42:55.499322+0000","last_fullsized":"2023-08-25T04:42:55.499322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:38.678165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038676+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:42:56.038676+0000","last_peered":"2023-08-25T04:42:56.038676+0000","last_clean":"2023-08-25T04:42:56.038676+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:42:56.038676+0000","last_undegraded":"2023-08-25T04:42:56.038676+0000","last_fullsized":"2023-08-25T04:42:56.038676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:10.325465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332224+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:42:55.332224+0000","last_peered":"2023-08-25T04:42:55.332224+0000","last_clean":"2023-08-25T04:42:55.332224+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:42:55.332224+0000","last_undegraded":"2023-08-25T04:42:55.332224+0000","last_fullsized":"2023-08-25T04:42:55.332224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:50:38.175235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499174+0000","last_change":"2023-08-25T04:42:50.329325+0000","last_active":"2023-08-25T04:42:55.499174+0000","last_peered":"2023-08-25T04:42:55.499174+0000","last_clean":"2023-08-25T04:42:55.499174+0000","last_became_active":"2023-08-25T04:42:50.329008+0000","last_became_peered":"2023-08-25T04:42:50.329008+0000","last_unstale":"2023-08-25T04:42:55.499174+0000","last_undegraded":"2023-08-25T04:42:55.499174+0000","last_fullsized":"2023-08-25T04:42:55.499174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:45.734751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038778+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:42:56.038778+0000","last_peered":"2023-08-25T04:42:56.038778+0000","last_clean":"2023-08-25T04:42:56.038778+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:42:56.038778+0000","last_undegraded":"2023-08-25T04:42:56.038778+0000","last_fullsized":"2023-08-25T04:42:56.038778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:50.076998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498529+0000","last_change":"2023-08-25T04:42:50.331990+0000","last_active":"2023-08-25T04:42:55.498529+0000","last_peered":"2023-08-25T04:42:55.498529+0000","last_clean":"2023-08-25T04:42:55.498529+0000","last_became_active":"2023-08-25T04:42:50.331614+0000","last_became_peered":"2023-08-25T04:42:50.331614+0000","last_unstale":"2023-08-25T04:42:55.498529+0000","last_undegraded":"2023-08-25T04:42:55.498529+0000","last_fullsized":"2023-08-25T04:42:55.498529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:48.466293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038263+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:42:56.038263+0000","last_peered":"2023-08-25T04:42:56.038263+0000","last_clean":"2023-08-25T04:42:56.038263+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:42:56.038263+0000","last_undegraded":"2023-08-25T04:42:56.038263+0000","last_fullsized":"2023-08-25T04:42:56.038263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:24.987336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332473+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:42:55.332473+0000","last_peered":"2023-08-25T04:42:55.332473+0000","last_clean":"2023-08-25T04:42:55.332473+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:42:55.332473+0000","last_undegraded":"2023-08-25T04:42:55.332473+0000","last_fullsized":"2023-08-25T04:42:55.332473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:47.251035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498372+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:42:55.498372+0000","last_peered":"2023-08-25T04:42:55.498372+0000","last_clean":"2023-08-25T04:42:55.498372+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:42:55.498372+0000","last_undegraded":"2023-08-25T04:42:55.498372+0000","last_fullsized":"2023-08-25T04:42:55.498372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:43:44.443815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499512+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:42:55.499512+0000","last_peered":"2023-08-25T04:42:55.499512+0000","last_clean":"2023-08-25T04:42:55.499512+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:42:55.499512+0000","last_undegraded":"2023-08-25T04:42:55.499512+0000","last_fullsized":"2023-08-25T04:42:55.499512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:53.310159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331924+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:42:55.331924+0000","last_peered":"2023-08-25T04:42:55.331924+0000","last_clean":"2023-08-25T04:42:55.331924+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:42:55.331924+0000","last_undegraded":"2023-08-25T04:42:55.331924+0000","last_fullsized":"2023-08-25T04:42:55.331924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:14.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499080+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:42:55.499080+0000","last_peered":"2023-08-25T04:42:55.499080+0000","last_clean":"2023-08-25T04:42:55.499080+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:42:55.499080+0000","last_undegraded":"2023-08-25T04:42:55.499080+0000","last_fullsized":"2023-08-25T04:42:55.499080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:03:40.257726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038256+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:42:56.038256+0000","last_peered":"2023-08-25T04:42:56.038256+0000","last_clean":"2023-08-25T04:42:56.038256+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:42:56.038256+0000","last_undegraded":"2023-08-25T04:42:56.038256+0000","last_fullsized":"2023-08-25T04:42:56.038256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:30:14.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332032+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:42:55.332032+0000","last_peered":"2023-08-25T04:42:55.332032+0000","last_clean":"2023-08-25T04:42:55.332032+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:42:55.332032+0000","last_undegraded":"2023-08-25T04:42:55.332032+0000","last_fullsized":"2023-08-25T04:42:55.332032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:36.560149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498804+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:42:55.498804+0000","last_peered":"2023-08-25T04:42:55.498804+0000","last_clean":"2023-08-25T04:42:55.498804+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:42:55.498804+0000","last_undegraded":"2023-08-25T04:42:55.498804+0000","last_fullsized":"2023-08-25T04:42:55.498804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:06.597833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332133+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:42:55.332133+0000","last_peered":"2023-08-25T04:42:55.332133+0000","last_clean":"2023-08-25T04:42:55.332133+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:42:55.332133+0000","last_undegraded":"2023-08-25T04:42:55.332133+0000","last_fullsized":"2023-08-25T04:42:55.332133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:32.544871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038780+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:42:56.038780+0000","last_peered":"2023-08-25T04:42:56.038780+0000","last_clean":"2023-08-25T04:42:56.038780+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:42:56.038780+0000","last_undegraded":"2023-08-25T04:42:56.038780+0000","last_fullsized":"2023-08-25T04:42:56.038780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:43:40.504423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497849+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:42:55.497849+0000","last_peered":"2023-08-25T04:42:55.497849+0000","last_clean":"2023-08-25T04:42:55.497849+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:42:55.497849+0000","last_undegraded":"2023-08-25T04:42:55.497849+0000","last_fullsized":"2023-08-25T04:42:55.497849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:51:52.545454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331405+0000","last_change":"2023-08-25T04:42:49.326537+0000","last_active":"2023-08-25T04:42:55.331405+0000","last_peered":"2023-08-25T04:42:55.331405+0000","last_clean":"2023-08-25T04:42:55.331405+0000","last_became_active":"2023-08-25T04:42:49.325361+0000","last_became_peered":"2023-08-25T04:42:49.325361+0000","last_unstale":"2023-08-25T04:42:55.331405+0000","last_undegraded":"2023-08-25T04:42:55.331405+0000","last_fullsized":"2023-08-25T04:42:55.331405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:49:55.273612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498837+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:42:55.498837+0000","last_peered":"2023-08-25T04:42:55.498837+0000","last_clean":"2023-08-25T04:42:55.498837+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:42:55.498837+0000","last_undegraded":"2023-08-25T04:42:55.498837+0000","last_fullsized":"2023-08-25T04:42:55.498837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:17.394975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499255+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:42:55.499255+0000","last_peered":"2023-08-25T04:42:55.499255+0000","last_clean":"2023-08-25T04:42:55.499255+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:42:55.499255+0000","last_undegraded":"2023-08-25T04:42:55.499255+0000","last_fullsized":"2023-08-25T04:42:55.499255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:03.796144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498750+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:42:55.498750+0000","last_peered":"2023-08-25T04:42:55.498750+0000","last_clean":"2023-08-25T04:42:55.498750+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:42:55.498750+0000","last_undegraded":"2023-08-25T04:42:55.498750+0000","last_fullsized":"2023-08-25T04:42:55.498750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:34.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498480+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:42:55.498480+0000","last_peered":"2023-08-25T04:42:55.498480+0000","last_clean":"2023-08-25T04:42:55.498480+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:42:55.498480+0000","last_undegraded":"2023-08-25T04:42:55.498480+0000","last_fullsized":"2023-08-25T04:42:55.498480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:22.278245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499059+0000","last_change":"2023-08-25T04:42:54.357813+0000","last_active":"2023-08-25T04:42:55.499059+0000","last_peered":"2023-08-25T04:42:55.499059+0000","last_clean":"2023-08-25T04:42:55.499059+0000","last_became_active":"2023-08-25T04:42:54.357400+0000","last_became_peered":"2023-08-25T04:42:54.357400+0000","last_unstale":"2023-08-25T04:42:55.499059+0000","last_undegraded":"2023-08-25T04:42:55.499059+0000","last_fullsized":"2023-08-25T04:42:55.499059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:35.674214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498104+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:42:55.498104+0000","last_peered":"2023-08-25T04:42:55.498104+0000","last_clean":"2023-08-25T04:42:55.498104+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:42:55.498104+0000","last_undegraded":"2023-08-25T04:42:55.498104+0000","last_fullsized":"2023-08-25T04:42:55.498104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:08.261819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331753+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:42:55.331753+0000","last_peered":"2023-08-25T04:42:55.331753+0000","last_clean":"2023-08-25T04:42:55.331753+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:42:55.331753+0000","last_undegraded":"2023-08-25T04:42:55.331753+0000","last_fullsized":"2023-08-25T04:42:55.331753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:45.227581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331399+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:42:55.331399+0000","last_peered":"2023-08-25T04:42:55.331399+0000","last_clean":"2023-08-25T04:42:55.331399+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:42:55.331399+0000","last_undegraded":"2023-08-25T04:42:55.331399+0000","last_fullsized":"2023-08-25T04:42:55.331399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:22:21.252880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498044+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:42:55.498044+0000","last_peered":"2023-08-25T04:42:55.498044+0000","last_clean":"2023-08-25T04:42:55.498044+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:42:55.498044+0000","last_undegraded":"2023-08-25T04:42:55.498044+0000","last_fullsized":"2023-08-25T04:42:55.498044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:02.689977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497914+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:42:55.497914+0000","last_peered":"2023-08-25T04:42:55.497914+0000","last_clean":"2023-08-25T04:42:55.497914+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:42:55.497914+0000","last_undegraded":"2023-08-25T04:42:55.497914+0000","last_fullsized":"2023-08-25T04:42:55.497914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:22.085043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498363+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:42:55.498363+0000","last_peered":"2023-08-25T04:42:55.498363+0000","last_clean":"2023-08-25T04:42:55.498363+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:42:55.498363+0000","last_undegraded":"2023-08-25T04:42:55.498363+0000","last_fullsized":"2023-08-25T04:42:55.498363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:36:10.911830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498328+0000","last_change":"2023-08-25T04:42:49.327175+0000","last_active":"2023-08-25T04:42:55.498328+0000","last_peered":"2023-08-25T04:42:55.498328+0000","last_clean":"2023-08-25T04:42:55.498328+0000","last_became_active":"2023-08-25T04:42:49.326884+0000","last_became_peered":"2023-08-25T04:42:49.326884+0000","last_unstale":"2023-08-25T04:42:55.498328+0000","last_undegraded":"2023-08-25T04:42:55.498328+0000","last_fullsized":"2023-08-25T04:42:55.498328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:57:48.362984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498436+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:42:55.498436+0000","last_peered":"2023-08-25T04:42:55.498436+0000","last_clean":"2023-08-25T04:42:55.498436+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:42:55.498436+0000","last_undegraded":"2023-08-25T04:42:55.498436+0000","last_fullsized":"2023-08-25T04:42:55.498436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:32:00.600660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498507+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:42:55.498507+0000","last_peered":"2023-08-25T04:42:55.498507+0000","last_clean":"2023-08-25T04:42:55.498507+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:42:55.498507+0000","last_undegraded":"2023-08-25T04:42:55.498507+0000","last_fullsized":"2023-08-25T04:42:55.498507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:34:51.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332068+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:42:55.332068+0000","last_peered":"2023-08-25T04:42:55.332068+0000","last_clean":"2023-08-25T04:42:55.332068+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:42:55.332068+0000","last_undegraded":"2023-08-25T04:42:55.332068+0000","last_fullsized":"2023-08-25T04:42:55.332068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:32:33.439250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332423+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:42:55.332423+0000","last_peered":"2023-08-25T04:42:55.332423+0000","last_clean":"2023-08-25T04:42:55.332423+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:42:55.332423+0000","last_undegraded":"2023-08-25T04:42:55.332423+0000","last_fullsized":"2023-08-25T04:42:55.332423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:18:35.106901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498795+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:42:55.498795+0000","last_peered":"2023-08-25T04:42:55.498795+0000","last_clean":"2023-08-25T04:42:55.498795+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:42:55.498795+0000","last_undegraded":"2023-08-25T04:42:55.498795+0000","last_fullsized":"2023-08-25T04:42:55.498795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:29.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498789+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:42:55.498789+0000","last_peered":"2023-08-25T04:42:55.498789+0000","last_clean":"2023-08-25T04:42:55.498789+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:42:55.498789+0000","last_undegraded":"2023-08-25T04:42:55.498789+0000","last_fullsized":"2023-08-25T04:42:55.498789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:30.087944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332775+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:42:55.332775+0000","last_peered":"2023-08-25T04:42:55.332775+0000","last_clean":"2023-08-25T04:42:55.332775+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:42:55.332775+0000","last_undegraded":"2023-08-25T04:42:55.332775+0000","last_fullsized":"2023-08-25T04:42:55.332775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:40:22.161009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497983+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:42:55.497983+0000","last_peered":"2023-08-25T04:42:55.497983+0000","last_clean":"2023-08-25T04:42:55.497983+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:42:55.497983+0000","last_undegraded":"2023-08-25T04:42:55.497983+0000","last_fullsized":"2023-08-25T04:42:55.497983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:17:20.821857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500462+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:42:55.500462+0000","last_peered":"2023-08-25T04:42:55.500462+0000","last_clean":"2023-08-25T04:42:55.500462+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:42:55.500462+0000","last_undegraded":"2023-08-25T04:42:55.500462+0000","last_fullsized":"2023-08-25T04:42:55.500462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:17.667227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332398+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:42:55.332398+0000","last_peered":"2023-08-25T04:42:55.332398+0000","last_clean":"2023-08-25T04:42:55.332398+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:42:55.332398+0000","last_undegraded":"2023-08-25T04:42:55.332398+0000","last_fullsized":"2023-08-25T04:42:55.332398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:21:30.344818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497999+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:42:55.497999+0000","last_peered":"2023-08-25T04:42:55.497999+0000","last_clean":"2023-08-25T04:42:55.497999+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:42:55.497999+0000","last_undegraded":"2023-08-25T04:42:55.497999+0000","last_fullsized":"2023-08-25T04:42:55.497999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:07:58.375742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038864+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:42:56.038864+0000","last_peered":"2023-08-25T04:42:56.038864+0000","last_clean":"2023-08-25T04:42:56.038864+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:42:56.038864+0000","last_undegraded":"2023-08-25T04:42:56.038864+0000","last_fullsized":"2023-08-25T04:42:56.038864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:18:13.540910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332682+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:42:55.332682+0000","last_peered":"2023-08-25T04:42:55.332682+0000","last_clean":"2023-08-25T04:42:55.332682+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:42:55.332682+0000","last_undegraded":"2023-08-25T04:42:55.332682+0000","last_fullsized":"2023-08-25T04:42:55.332682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:42.788095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498068+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:42:55.498068+0000","last_peered":"2023-08-25T04:42:55.498068+0000","last_clean":"2023-08-25T04:42:55.498068+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:42:55.498068+0000","last_undegraded":"2023-08-25T04:42:55.498068+0000","last_fullsized":"2023-08-25T04:42:55.498068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:39.945056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499361+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:42:55.499361+0000","last_peered":"2023-08-25T04:42:55.499361+0000","last_clean":"2023-08-25T04:42:55.499361+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:42:55.499361+0000","last_undegraded":"2023-08-25T04:42:55.499361+0000","last_fullsized":"2023-08-25T04:42:55.499361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:28:28.952965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498165+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:42:55.498165+0000","last_peered":"2023-08-25T04:42:55.498165+0000","last_clean":"2023-08-25T04:42:55.498165+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:42:55.498165+0000","last_undegraded":"2023-08-25T04:42:55.498165+0000","last_fullsized":"2023-08-25T04:42:55.498165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:05.052741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498192+0000","last_change":"2023-08-25T04:42:50.330514+0000","last_active":"2023-08-25T04:42:55.498192+0000","last_peered":"2023-08-25T04:42:55.498192+0000","last_clean":"2023-08-25T04:42:55.498192+0000","last_became_active":"2023-08-25T04:42:50.330001+0000","last_became_peered":"2023-08-25T04:42:50.330001+0000","last_unstale":"2023-08-25T04:42:55.498192+0000","last_undegraded":"2023-08-25T04:42:55.498192+0000","last_fullsized":"2023-08-25T04:42:55.498192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:21.145109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038823+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:42:56.038823+0000","last_peered":"2023-08-25T04:42:56.038823+0000","last_clean":"2023-08-25T04:42:56.038823+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:42:56.038823+0000","last_undegraded":"2023-08-25T04:42:56.038823+0000","last_fullsized":"2023-08-25T04:42:56.038823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:35:11.398547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499668+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:42:55.499668+0000","last_peered":"2023-08-25T04:42:55.499668+0000","last_clean":"2023-08-25T04:42:55.499668+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:42:55.499668+0000","last_undegraded":"2023-08-25T04:42:55.499668+0000","last_fullsized":"2023-08-25T04:42:55.499668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:32.084873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499048+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:42:55.499048+0000","last_peered":"2023-08-25T04:42:55.499048+0000","last_clean":"2023-08-25T04:42:55.499048+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:42:55.499048+0000","last_undegraded":"2023-08-25T04:42:55.499048+0000","last_fullsized":"2023-08-25T04:42:55.499048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:05:26.906447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498927+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:42:55.498927+0000","last_peered":"2023-08-25T04:42:55.498927+0000","last_clean":"2023-08-25T04:42:55.498927+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:42:55.498927+0000","last_undegraded":"2023-08-25T04:42:55.498927+0000","last_fullsized":"2023-08-25T04:42:55.498927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:46.733202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498390+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:42:55.498390+0000","last_peered":"2023-08-25T04:42:55.498390+0000","last_clean":"2023-08-25T04:42:55.498390+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:42:55.498390+0000","last_undegraded":"2023-08-25T04:42:55.498390+0000","last_fullsized":"2023-08-25T04:42:55.498390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:50:23.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498165+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:42:55.498165+0000","last_peered":"2023-08-25T04:42:55.498165+0000","last_clean":"2023-08-25T04:42:55.498165+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:42:55.498165+0000","last_undegraded":"2023-08-25T04:42:55.498165+0000","last_fullsized":"2023-08-25T04:42:55.498165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:26.391472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499133+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:42:55.499133+0000","last_peered":"2023-08-25T04:42:55.499133+0000","last_clean":"2023-08-25T04:42:55.499133+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:42:55.499133+0000","last_undegraded":"2023-08-25T04:42:55.499133+0000","last_fullsized":"2023-08-25T04:42:55.499133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:09:50.780046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038634+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:42:56.038634+0000","last_peered":"2023-08-25T04:42:56.038634+0000","last_clean":"2023-08-25T04:42:56.038634+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:42:56.038634+0000","last_undegraded":"2023-08-25T04:42:56.038634+0000","last_fullsized":"2023-08-25T04:42:56.038634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:33.955508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498727+0000","last_change":"2023-08-25T04:42:50.327235+0000","last_active":"2023-08-25T04:42:55.498727+0000","last_peered":"2023-08-25T04:42:55.498727+0000","last_clean":"2023-08-25T04:42:55.498727+0000","last_became_active":"2023-08-25T04:42:50.326486+0000","last_became_peered":"2023-08-25T04:42:50.326486+0000","last_unstale":"2023-08-25T04:42:55.498727+0000","last_undegraded":"2023-08-25T04:42:55.498727+0000","last_fullsized":"2023-08-25T04:42:55.498727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:26.595739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":15308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":463755,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79300000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.0249999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.90200000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1058159,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.443},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.629},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.80900000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78400000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":464337,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.62}]},{"osd":1,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.505},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.748}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103197,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":464028,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.622}]},{"osd":1,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.29599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.498}]},{"osd":3,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77800000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.98199999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.87},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1054374,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.41899999999999998,"15min":0.40500000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86799999999999999,"5min":1.026,"15min":1.026},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.53500000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":0.996,"15min":0.996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93200000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58599999999999997}]},{"osd":9,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60799999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463755,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.119,"5min":1.131,"15min":1.131},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.68600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.68300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.177,"5min":1.389,"15min":1.389},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.752},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.874},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":467324,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83199999999999996,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85299999999999998,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.63}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54800000000000004,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.54200000000000004,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.153,"5min":1.548,"15min":1.548},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.441,"15min":0.41599999999999998},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.503}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":1.613,"15min":1.613},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89700000000000002,"5min":1.597,"15min":1.597},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Aug 25 04:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92000000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:42:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":465233,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.44600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.373,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.77100000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.56899999999999995},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":1.194,"15min":1.194},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81699999999999995,"5min":1.488,"15min":1.488},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.99399999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94299999999999995,"5min":1.603,"15min":1.603},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1930000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1599999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.97999999999999998,"5min":1.534,"15min":1.534},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.751,"15min":0.755},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92000000000000004,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.96599999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.056,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0840000000000001,"5min":1.365,"15min":1.365},"last":0.874}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.002}]},{"osd":10,"last update":"Fri Aug 25 04:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri Aug 25 04:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.01},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.94199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":467631,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76000000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63}]},{"osd":1,"last update":"Fri Aug 25 04:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.53100000000000003,"15min":0.501},"min":{"1min":0.40600000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.879,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:42:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.878},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Aug 25 04:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81899999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.0569999999999999}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.92000000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.071},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":1.04}]},{"osd":10,"last update":"Fri Aug 25 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.091},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.0249999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":463859,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41699999999999998},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.505,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.004,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.751},{"interface":"front","average":{"1min":0.748,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.57099999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":1.53,"15min":1.53},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":1.573,"15min":1.573},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.621},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2370000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92500000000000004,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85299999999999998,"5min":1.556,"15min":1.556},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.98199999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.73499999999999999,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.69099999999999995}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.879}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920814,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463777,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.36599999999999999,"15min":0.34599999999999997},"min":{"1min":0.30299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.378,"15min":0.36099999999999999},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84999999999999998,"5min":0.876,"15min":0.876},"last":0.499}]},{"osd":3,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.432},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69599999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93400000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.59899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1054305,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.53600000000000003,"15min":0.5},"min":{"1min":0.432,"5min":0.318,"15min":0.318},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.30299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.371},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.39400000000000002},"min":{"1min":0.30499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.624},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.503},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57199999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.749},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:43:17.966 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T04:43:18.052 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:17 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1064036058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.053 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:17 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/483627766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:17 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1064036058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:17 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/483627766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:17 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1064036058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:17 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/483627766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 -- 172.21.15.145:0/3167918282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c101b10 msgr2=0x7f4f1c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3167918282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c101b10 0x7f4f1c101f70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f100099c0 tx=0x7f4f10009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 -- 172.21.15.145:0/3167918282 shutdown_connections 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3167918282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3167918282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c100920 0x7f4f1c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3167918282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c101b10 0x7f4f1c101f70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 -- 172.21.15.145:0/3167918282 >> 172.21.15.145:0/3167918282 conn(0x7f4f1c0fbe50 msgr2=0x7f4f1c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:18.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 -- 172.21.15.145:0/3167918282 shutdown_connections 2023-08-25T04:43:18.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.189+0000 7f4f221ff700 1 -- 172.21.15.145:0/3167918282 wait complete. 2023-08-25T04:43:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 Processor -- start 2023-08-25T04:43:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 -- start start 2023-08-25T04:43:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c101b10 0x7f4f1c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19e140 con 0x7f4f1c100920 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19e280 con 0x7f4f1c1024b0 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f221ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19e410 con 0x7f4f1c101b10 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.191+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56684/0 (socket says 172.21.15.145:56684) 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 -- 172.21.15.145:0/3488617679 learned_addr learned my addr 172.21.15.145:0/3488617679 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1affd700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c101b10 0x7f4f1c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1bfff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 -- 172.21.15.145:0/3488617679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c101b10 msgr2=0x7f4f1c196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c101b10 0x7f4f1c196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 -- 172.21.15.145:0/3488617679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 msgr2=0x7f4f1c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 -- 172.21.15.145:0/3488617679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100095f0 con 0x7f4f1c100920 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1bfff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f1b7fe700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e970 tx=0x7f4f0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:18.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c009790 con 0x7f4f1c100920 2023-08-25T04:43:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c19e6f0 con 0x7f4f1c100920 2023-08-25T04:43:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c00eec0 con 0x7f4f1c100920 2023-08-25T04:43:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.192+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003cf0 con 0x7f4f1c100920 2023-08-25T04:43:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.193+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c19ec00 con 0x7f4f1c100920 2023-08-25T04:43:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.194+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c10c130 con 0x7f4f1c100920 2023-08-25T04:43:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.196+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4f0c00ccd0 con 0x7f4f1c100920 2023-08-25T04:43:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.198+0000 7f4f18ff9700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 0x7f4f040a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.198+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4f0c014070 con 0x7f4f1c100920 2023-08-25T04:43:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.198+0000 7f4f1affd700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 0x7f4f040a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.199+0000 7f4f1affd700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 0x7f4f040a2460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b580 tx=0x7f4f10005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:18.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.202+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4f0c095230 con 0x7f4f1c100920 2023-08-25T04:43:18.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.404+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f1c060d60 con 0x7f4f0409ffa0 2023-08-25T04:43:18.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.416+0000 7f4f18ff9700 1 -- 172.21.15.145:0/3488617679 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422435 (secure 0 0 0) 0x7f4f1c060d60 con 0x7f4f0409ffa0 2023-08-25T04:43:18.424 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:18.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.421+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 msgr2=0x7f4f040a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:18.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.421+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 0x7f4f040a2460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b580 tx=0x7f4f10005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.421+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 msgr2=0x7f4f1c1965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:18.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.421+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e970 tx=0x7f4f0c00ec80 comp rx=0 tx=0).stop 2023-08-25T04:43:18.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 shutdown_connections 2023-08-25T04:43:18.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f0409ffa0 0x7f4f040a2460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c1024b0 0x7f4f1c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f1c100920 0x7f4f1c1965c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 --2- 172.21.15.145:0/3488617679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f1c101b10 0x7f4f1c196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 >> 172.21.15.145:0/3488617679 conn(0x7f4f1c0fbe50 msgr2=0x7f4f1c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 shutdown_connections 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:18.422+0000 7f4f221ff700 1 -- 172.21.15.145:0/3488617679 wait complete. 2023-08-25T04:43:18.430 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:43:18.443 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-08-25T04:43:17.689930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":230,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":180,"ondisk_log_size":180,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335492,"kb_used_data":11244,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606332,"statfs":{"total":1151940427776,"available":1151596883968,"internally_reserved":0,"allocated":11513856,"data_stored":7350537,"data_compressed":247383,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003390"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498548+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:42:55.498548+0000","last_peered":"2023-08-25T04:42:55.498548+0000","last_clean":"2023-08-25T04:42:55.498548+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:42:55.498548+0000","last_undegraded":"2023-08-25T04:42:55.498548+0000","last_fullsized":"2023-08-25T04:42:55.498548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:11.466599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498351+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:42:55.498351+0000","last_peered":"2023-08-25T04:42:55.498351+0000","last_clean":"2023-08-25T04:42:55.498351+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:42:55.498351+0000","last_undegraded":"2023-08-25T04:42:55.498351+0000","last_fullsized":"2023-08-25T04:42:55.498351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:40.571648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498628+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:42:55.498628+0000","last_peered":"2023-08-25T04:42:55.498628+0000","last_clean":"2023-08-25T04:42:55.498628+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:42:55.498628+0000","last_undegraded":"2023-08-25T04:42:55.498628+0000","last_fullsized":"2023-08-25T04:42:55.498628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:23.547350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498876+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:42:55.498876+0000","last_peered":"2023-08-25T04:42:55.498876+0000","last_clean":"2023-08-25T04:42:55.498876+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:42:55.498876+0000","last_undegraded":"2023-08-25T04:42:55.498876+0000","last_fullsized":"2023-08-25T04:42:55.498876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:12.102282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331964+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:42:55.331964+0000","last_peered":"2023-08-25T04:42:55.331964+0000","last_clean":"2023-08-25T04:42:55.331964+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:42:55.331964+0000","last_undegraded":"2023-08-25T04:42:55.331964+0000","last_fullsized":"2023-08-25T04:42:55.331964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:11.148176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499676+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:42:55.499676+0000","last_peered":"2023-08-25T04:42:55.499676+0000","last_clean":"2023-08-25T04:42:55.499676+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:42:55.499676+0000","last_undegraded":"2023-08-25T04:42:55.499676+0000","last_fullsized":"2023-08-25T04:42:55.499676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:49.556742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332092+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:42:55.332092+0000","last_peered":"2023-08-25T04:42:55.332092+0000","last_clean":"2023-08-25T04:42:55.332092+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:42:55.332092+0000","last_undegraded":"2023-08-25T04:42:55.332092+0000","last_fullsized":"2023-08-25T04:42:55.332092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:02.949033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497903+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:42:55.497903+0000","last_peered":"2023-08-25T04:42:55.497903+0000","last_clean":"2023-08-25T04:42:55.497903+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:42:55.497903+0000","last_undegraded":"2023-08-25T04:42:55.497903+0000","last_fullsized":"2023-08-25T04:42:55.497903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:51:08.670149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498711+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:42:55.498711+0000","last_peered":"2023-08-25T04:42:55.498711+0000","last_clean":"2023-08-25T04:42:55.498711+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:42:55.498711+0000","last_undegraded":"2023-08-25T04:42:55.498711+0000","last_fullsized":"2023-08-25T04:42:55.498711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:50:22.640028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499835+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:42:55.499835+0000","last_peered":"2023-08-25T04:42:55.499835+0000","last_clean":"2023-08-25T04:42:55.499835+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:42:55.499835+0000","last_undegraded":"2023-08-25T04:42:55.499835+0000","last_fullsized":"2023-08-25T04:42:55.499835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:12:37.196016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332636+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:42:55.332636+0000","last_peered":"2023-08-25T04:42:55.332636+0000","last_clean":"2023-08-25T04:42:55.332636+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:42:55.332636+0000","last_undegraded":"2023-08-25T04:42:55.332636+0000","last_fullsized":"2023-08-25T04:42:55.332636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:30.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498731+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:42:55.498731+0000","last_peered":"2023-08-25T04:42:55.498731+0000","last_clean":"2023-08-25T04:42:55.498731+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:42:55.498731+0000","last_undegraded":"2023-08-25T04:42:55.498731+0000","last_fullsized":"2023-08-25T04:42:55.498731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:42.786018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038549+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:42:56.038549+0000","last_peered":"2023-08-25T04:42:56.038549+0000","last_clean":"2023-08-25T04:42:56.038549+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:42:56.038549+0000","last_undegraded":"2023-08-25T04:42:56.038549+0000","last_fullsized":"2023-08-25T04:42:56.038549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:16:22.636180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498103+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:42:55.498103+0000","last_peered":"2023-08-25T04:42:55.498103+0000","last_clean":"2023-08-25T04:42:55.498103+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:42:55.498103+0000","last_undegraded":"2023-08-25T04:42:55.498103+0000","last_fullsized":"2023-08-25T04:42:55.498103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:04.859467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498197+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:42:55.498197+0000","last_peered":"2023-08-25T04:42:55.498197+0000","last_clean":"2023-08-25T04:42:55.498197+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:42:55.498197+0000","last_undegraded":"2023-08-25T04:42:55.498197+0000","last_fullsized":"2023-08-25T04:42:55.498197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:08.029332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499673+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:42:55.499673+0000","last_peered":"2023-08-25T04:42:55.499673+0000","last_clean":"2023-08-25T04:42:55.499673+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:42:55.499673+0000","last_undegraded":"2023-08-25T04:42:55.499673+0000","last_fullsized":"2023-08-25T04:42:55.499673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:03.011607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498394+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:42:55.498394+0000","last_peered":"2023-08-25T04:42:55.498394+0000","last_clean":"2023-08-25T04:42:55.498394+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:42:55.498394+0000","last_undegraded":"2023-08-25T04:42:55.498394+0000","last_fullsized":"2023-08-25T04:42:55.498394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:09.299405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331350+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:42:55.331350+0000","last_peered":"2023-08-25T04:42:55.331350+0000","last_clean":"2023-08-25T04:42:55.331350+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:42:55.331350+0000","last_undegraded":"2023-08-25T04:42:55.331350+0000","last_fullsized":"2023-08-25T04:42:55.331350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:47.697095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499136+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:42:55.499136+0000","last_peered":"2023-08-25T04:42:55.499136+0000","last_clean":"2023-08-25T04:42:55.499136+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:42:55.499136+0000","last_undegraded":"2023-08-25T04:42:55.499136+0000","last_fullsized":"2023-08-25T04:42:55.499136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:03.430027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499400+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:42:55.499400+0000","last_peered":"2023-08-25T04:42:55.499400+0000","last_clean":"2023-08-25T04:42:55.499400+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:42:55.499400+0000","last_undegraded":"2023-08-25T04:42:55.499400+0000","last_fullsized":"2023-08-25T04:42:55.499400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:26.766631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331429+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:42:55.331429+0000","last_peered":"2023-08-25T04:42:55.331429+0000","last_clean":"2023-08-25T04:42:55.331429+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:42:55.331429+0000","last_undegraded":"2023-08-25T04:42:55.331429+0000","last_fullsized":"2023-08-25T04:42:55.331429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:49:03.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498335+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:42:55.498335+0000","last_peered":"2023-08-25T04:42:55.498335+0000","last_clean":"2023-08-25T04:42:55.498335+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:42:55.498335+0000","last_undegraded":"2023-08-25T04:42:55.498335+0000","last_fullsized":"2023-08-25T04:42:55.498335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:38:30.988002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498343+0000","last_change":"2023-08-25T04:42:54.362609+0000","last_active":"2023-08-25T04:42:55.498343+0000","last_peered":"2023-08-25T04:42:55.498343+0000","last_clean":"2023-08-25T04:42:55.498343+0000","last_became_active":"2023-08-25T04:42:54.362224+0000","last_became_peered":"2023-08-25T04:42:54.362224+0000","last_unstale":"2023-08-25T04:42:55.498343+0000","last_undegraded":"2023-08-25T04:42:55.498343+0000","last_fullsized":"2023-08-25T04:42:55.498343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:21:07.957008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498561+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:42:55.498561+0000","last_peered":"2023-08-25T04:42:55.498561+0000","last_clean":"2023-08-25T04:42:55.498561+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:42:55.498561+0000","last_undegraded":"2023-08-25T04:42:55.498561+0000","last_fullsized":"2023-08-25T04:42:55.498561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:04.868337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498291+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:42:55.498291+0000","last_peered":"2023-08-25T04:42:55.498291+0000","last_clean":"2023-08-25T04:42:55.498291+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:42:55.498291+0000","last_undegraded":"2023-08-25T04:42:55.498291+0000","last_fullsized":"2023-08-25T04:42:55.498291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:50.565716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499241+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:42:55.499241+0000","last_peered":"2023-08-25T04:42:55.499241+0000","last_clean":"2023-08-25T04:42:55.499241+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:42:55.499241+0000","last_undegraded":"2023-08-25T04:42:55.499241+0000","last_fullsized":"2023-08-25T04:42:55.499241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:14.665503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332300+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:42:55.332300+0000","last_peered":"2023-08-25T04:42:55.332300+0000","last_clean":"2023-08-25T04:42:55.332300+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:42:55.332300+0000","last_undegraded":"2023-08-25T04:42:55.332300+0000","last_fullsized":"2023-08-25T04:42:55.332300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:00:31.012805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498445+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:42:55.498445+0000","last_peered":"2023-08-25T04:42:55.498445+0000","last_clean":"2023-08-25T04:42:55.498445+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:42:55.498445+0000","last_undegraded":"2023-08-25T04:42:55.498445+0000","last_fullsized":"2023-08-25T04:42:55.498445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:20:46.135430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498850+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:42:55.498850+0000","last_peered":"2023-08-25T04:42:55.498850+0000","last_clean":"2023-08-25T04:42:55.498850+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:42:55.498850+0000","last_undegraded":"2023-08-25T04:42:55.498850+0000","last_fullsized":"2023-08-25T04:42:55.498850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:22:03.190298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332210+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:42:55.332210+0000","last_peered":"2023-08-25T04:42:55.332210+0000","last_clean":"2023-08-25T04:42:55.332210+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:42:55.332210+0000","last_undegraded":"2023-08-25T04:42:55.332210+0000","last_fullsized":"2023-08-25T04:42:55.332210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:10.908437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500392+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:42:55.500392+0000","last_peered":"2023-08-25T04:42:55.500392+0000","last_clean":"2023-08-25T04:42:55.500392+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:42:55.500392+0000","last_undegraded":"2023-08-25T04:42:55.500392+0000","last_fullsized":"2023-08-25T04:42:55.500392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:13:17.680357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498603+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:42:55.498603+0000","last_peered":"2023-08-25T04:42:55.498603+0000","last_clean":"2023-08-25T04:42:55.498603+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:42:55.498603+0000","last_undegraded":"2023-08-25T04:42:55.498603+0000","last_fullsized":"2023-08-25T04:42:55.498603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:29.499407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498941+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:42:55.498941+0000","last_peered":"2023-08-25T04:42:55.498941+0000","last_clean":"2023-08-25T04:42:55.498941+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:42:55.498941+0000","last_undegraded":"2023-08-25T04:42:55.498941+0000","last_fullsized":"2023-08-25T04:42:55.498941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:45.183593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332275+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:42:55.332275+0000","last_peered":"2023-08-25T04:42:55.332275+0000","last_clean":"2023-08-25T04:42:55.332275+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:42:55.332275+0000","last_undegraded":"2023-08-25T04:42:55.332275+0000","last_fullsized":"2023-08-25T04:42:55.332275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:39.960781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498543+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:42:55.498543+0000","last_peered":"2023-08-25T04:42:55.498543+0000","last_clean":"2023-08-25T04:42:55.498543+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:42:55.498543+0000","last_undegraded":"2023-08-25T04:42:55.498543+0000","last_fullsized":"2023-08-25T04:42:55.498543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:58:01.500963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499498+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:42:55.499498+0000","last_peered":"2023-08-25T04:42:55.499498+0000","last_clean":"2023-08-25T04:42:55.499498+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:42:55.499498+0000","last_undegraded":"2023-08-25T04:42:55.499498+0000","last_fullsized":"2023-08-25T04:42:55.499498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:48.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499063+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:42:55.499063+0000","last_peered":"2023-08-25T04:42:55.499063+0000","last_clean":"2023-08-25T04:42:55.499063+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:42:55.499063+0000","last_undegraded":"2023-08-25T04:42:55.499063+0000","last_fullsized":"2023-08-25T04:42:55.499063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:26.652793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498234+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:42:55.498234+0000","last_peered":"2023-08-25T04:42:55.498234+0000","last_clean":"2023-08-25T04:42:55.498234+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:42:55.498234+0000","last_undegraded":"2023-08-25T04:42:55.498234+0000","last_fullsized":"2023-08-25T04:42:55.498234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:18.199795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038361+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:42:56.038361+0000","last_peered":"2023-08-25T04:42:56.038361+0000","last_clean":"2023-08-25T04:42:56.038361+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:42:56.038361+0000","last_undegraded":"2023-08-25T04:42:56.038361+0000","last_fullsized":"2023-08-25T04:42:56.038361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:28.752707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497778+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:42:55.497778+0000","last_peered":"2023-08-25T04:42:55.497778+0000","last_clean":"2023-08-25T04:42:55.497778+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:42:55.497778+0000","last_undegraded":"2023-08-25T04:42:55.497778+0000","last_fullsized":"2023-08-25T04:42:55.497778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:35:07.264982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498249+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:42:55.498249+0000","last_peered":"2023-08-25T04:42:55.498249+0000","last_clean":"2023-08-25T04:42:55.498249+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:42:55.498249+0000","last_undegraded":"2023-08-25T04:42:55.498249+0000","last_fullsized":"2023-08-25T04:42:55.498249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:12.166934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498231+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:42:55.498231+0000","last_peered":"2023-08-25T04:42:55.498231+0000","last_clean":"2023-08-25T04:42:55.498231+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:42:55.498231+0000","last_undegraded":"2023-08-25T04:42:55.498231+0000","last_fullsized":"2023-08-25T04:42:55.498231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:51.295038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330846+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:42:55.330846+0000","last_peered":"2023-08-25T04:42:55.330846+0000","last_clean":"2023-08-25T04:42:55.330846+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:42:55.330846+0000","last_undegraded":"2023-08-25T04:42:55.330846+0000","last_fullsized":"2023-08-25T04:42:55.330846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:47.527846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497962+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:42:55.497962+0000","last_peered":"2023-08-25T04:42:55.497962+0000","last_clean":"2023-08-25T04:42:55.497962+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:42:55.497962+0000","last_undegraded":"2023-08-25T04:42:55.497962+0000","last_fullsized":"2023-08-25T04:42:55.497962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:56:30.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499596+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:42:55.499596+0000","last_peered":"2023-08-25T04:42:55.499596+0000","last_clean":"2023-08-25T04:42:55.499596+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:42:55.499596+0000","last_undegraded":"2023-08-25T04:42:55.499596+0000","last_fullsized":"2023-08-25T04:42:55.499596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:44.399055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498401+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:42:55.498401+0000","last_peered":"2023-08-25T04:42:55.498401+0000","last_clean":"2023-08-25T04:42:55.498401+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:42:55.498401+0000","last_undegraded":"2023-08-25T04:42:55.498401+0000","last_fullsized":"2023-08-25T04:42:55.498401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:05:40.395184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330846+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:42:55.330846+0000","last_peered":"2023-08-25T04:42:55.330846+0000","last_clean":"2023-08-25T04:42:55.330846+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:42:55.330846+0000","last_undegraded":"2023-08-25T04:42:55.330846+0000","last_fullsized":"2023-08-25T04:42:55.330846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:19:16.538915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332437+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:42:55.332437+0000","last_peered":"2023-08-25T04:42:55.332437+0000","last_clean":"2023-08-25T04:42:55.332437+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:42:55.332437+0000","last_undegraded":"2023-08-25T04:42:55.332437+0000","last_fullsized":"2023-08-25T04:42:55.332437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:49:29.145032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498462+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:42:55.498462+0000","last_peered":"2023-08-25T04:42:55.498462+0000","last_clean":"2023-08-25T04:42:55.498462+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:42:55.498462+0000","last_undegraded":"2023-08-25T04:42:55.498462+0000","last_fullsized":"2023-08-25T04:42:55.498462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:22.343180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499835+0000","last_change":"2023-08-25T04:42:54.357939+0000","last_active":"2023-08-25T04:42:55.499835+0000","last_peered":"2023-08-25T04:42:55.499835+0000","last_clean":"2023-08-25T04:42:55.499835+0000","last_became_active":"2023-08-25T04:42:54.357449+0000","last_became_peered":"2023-08-25T04:42:54.357449+0000","last_unstale":"2023-08-25T04:42:55.499835+0000","last_undegraded":"2023-08-25T04:42:55.499835+0000","last_fullsized":"2023-08-25T04:42:55.499835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:20:54.583324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500361+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:42:55.500361+0000","last_peered":"2023-08-25T04:42:55.500361+0000","last_clean":"2023-08-25T04:42:55.500361+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:42:55.500361+0000","last_undegraded":"2023-08-25T04:42:55.500361+0000","last_fullsized":"2023-08-25T04:42:55.500361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:17.153473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498682+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:42:55.498682+0000","last_peered":"2023-08-25T04:42:55.498682+0000","last_clean":"2023-08-25T04:42:55.498682+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:42:55.498682+0000","last_undegraded":"2023-08-25T04:42:55.498682+0000","last_fullsized":"2023-08-25T04:42:55.498682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:30:28.593283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.330967+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:42:55.330967+0000","last_peered":"2023-08-25T04:42:55.330967+0000","last_clean":"2023-08-25T04:42:55.330967+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:42:55.330967+0000","last_undegraded":"2023-08-25T04:42:55.330967+0000","last_fullsized":"2023-08-25T04:42:55.330967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:35.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'10","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:43:05.979386+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:43:05.979386+0000","last_peered":"2023-08-25T04:43:05.979386+0000","last_clean":"2023-08-25T04:43:05.979386+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:43:05.979386+0000","last_undegraded":"2023-08-25T04:43:05.979386+0000","last_fullsized":"2023-08-25T04:43:05.979386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:14.984411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332094+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:42:55.332094+0000","last_peered":"2023-08-25T04:42:55.332094+0000","last_clean":"2023-08-25T04:42:55.332094+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:42:55.332094+0000","last_undegraded":"2023-08-25T04:42:55.332094+0000","last_fullsized":"2023-08-25T04:42:55.332094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:17:28.690442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500617+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:42:55.500617+0000","last_peered":"2023-08-25T04:42:55.500617+0000","last_clean":"2023-08-25T04:42:55.500617+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:42:55.500617+0000","last_undegraded":"2023-08-25T04:42:55.500617+0000","last_fullsized":"2023-08-25T04:42:55.500617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:15.202392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498872+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:42:55.498872+0000","last_peered":"2023-08-25T04:42:55.498872+0000","last_clean":"2023-08-25T04:42:55.498872+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:42:55.498872+0000","last_undegraded":"2023-08-25T04:42:55.498872+0000","last_fullsized":"2023-08-25T04:42:55.498872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:27:58.673573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038450+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:42:56.038450+0000","last_peered":"2023-08-25T04:42:56.038450+0000","last_clean":"2023-08-25T04:42:56.038450+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:42:56.038450+0000","last_undegraded":"2023-08-25T04:42:56.038450+0000","last_fullsized":"2023-08-25T04:42:56.038450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:37.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498542+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:42:55.498542+0000","last_peered":"2023-08-25T04:42:55.498542+0000","last_clean":"2023-08-25T04:42:55.498542+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:42:55.498542+0000","last_undegraded":"2023-08-25T04:42:55.498542+0000","last_fullsized":"2023-08-25T04:42:55.498542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:31.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498649+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:42:55.498649+0000","last_peered":"2023-08-25T04:42:55.498649+0000","last_clean":"2023-08-25T04:42:55.498649+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:42:55.498649+0000","last_undegraded":"2023-08-25T04:42:55.498649+0000","last_fullsized":"2023-08-25T04:42:55.498649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:20.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332674+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:42:55.332674+0000","last_peered":"2023-08-25T04:42:55.332674+0000","last_clean":"2023-08-25T04:42:55.332674+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:42:55.332674+0000","last_undegraded":"2023-08-25T04:42:55.332674+0000","last_fullsized":"2023-08-25T04:42:55.332674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:49.143880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331296+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:42:55.331296+0000","last_peered":"2023-08-25T04:42:55.331296+0000","last_clean":"2023-08-25T04:42:55.331296+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:42:55.331296+0000","last_undegraded":"2023-08-25T04:42:55.331296+0000","last_fullsized":"2023-08-25T04:42:55.331296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:00:30.917396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499926+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:42:55.499926+0000","last_peered":"2023-08-25T04:42:55.499926+0000","last_clean":"2023-08-25T04:42:55.499926+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:42:55.499926+0000","last_undegraded":"2023-08-25T04:42:55.499926+0000","last_fullsized":"2023-08-25T04:42:55.499926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:59:16.962246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332401+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:42:55.332401+0000","last_peered":"2023-08-25T04:42:55.332401+0000","last_clean":"2023-08-25T04:42:55.332401+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:42:55.332401+0000","last_undegraded":"2023-08-25T04:42:55.332401+0000","last_fullsized":"2023-08-25T04:42:55.332401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:13:40.041773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038956+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:42:56.038956+0000","last_peered":"2023-08-25T04:42:56.038956+0000","last_clean":"2023-08-25T04:42:56.038956+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:42:56.038956+0000","last_undegraded":"2023-08-25T04:42:56.038956+0000","last_fullsized":"2023-08-25T04:42:56.038956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:25.874788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331742+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:42:55.331742+0000","last_peered":"2023-08-25T04:42:55.331742+0000","last_clean":"2023-08-25T04:42:55.331742+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:42:55.331742+0000","last_undegraded":"2023-08-25T04:42:55.331742+0000","last_fullsized":"2023-08-25T04:42:55.331742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:05:47.949354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500166+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:42:55.500166+0000","last_peered":"2023-08-25T04:42:55.500166+0000","last_clean":"2023-08-25T04:42:55.500166+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:42:55.500166+0000","last_undegraded":"2023-08-25T04:42:55.500166+0000","last_fullsized":"2023-08-25T04:42:55.500166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:09:57.787096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500037+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:42:55.500037+0000","last_peered":"2023-08-25T04:42:55.500037+0000","last_clean":"2023-08-25T04:42:55.500037+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:42:55.500037+0000","last_undegraded":"2023-08-25T04:42:55.500037+0000","last_fullsized":"2023-08-25T04:42:55.500037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:43:06.007419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332208+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:42:55.332208+0000","last_peered":"2023-08-25T04:42:55.332208+0000","last_clean":"2023-08-25T04:42:55.332208+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:42:55.332208+0000","last_undegraded":"2023-08-25T04:42:55.332208+0000","last_fullsized":"2023-08-25T04:42:55.332208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:20:47.744053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498670+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:42:55.498670+0000","last_peered":"2023-08-25T04:42:55.498670+0000","last_clean":"2023-08-25T04:42:55.498670+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:42:55.498670+0000","last_undegraded":"2023-08-25T04:42:55.498670+0000","last_fullsized":"2023-08-25T04:42:55.498670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:37.815247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331991+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:42:55.331991+0000","last_peered":"2023-08-25T04:42:55.331991+0000","last_clean":"2023-08-25T04:42:55.331991+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:42:55.331991+0000","last_undegraded":"2023-08-25T04:42:55.331991+0000","last_fullsized":"2023-08-25T04:42:55.331991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:13.490903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331139+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:42:55.331139+0000","last_peered":"2023-08-25T04:42:55.331139+0000","last_clean":"2023-08-25T04:42:55.331139+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:42:55.331139+0000","last_undegraded":"2023-08-25T04:42:55.331139+0000","last_fullsized":"2023-08-25T04:42:55.331139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:01.814559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499322+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:42:55.499322+0000","last_peered":"2023-08-25T04:42:55.499322+0000","last_clean":"2023-08-25T04:42:55.499322+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:42:55.499322+0000","last_undegraded":"2023-08-25T04:42:55.499322+0000","last_fullsized":"2023-08-25T04:42:55.499322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:38.678165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038676+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:42:56.038676+0000","last_peered":"2023-08-25T04:42:56.038676+0000","last_clean":"2023-08-25T04:42:56.038676+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:42:56.038676+0000","last_undegraded":"2023-08-25T04:42:56.038676+0000","last_fullsized":"2023-08-25T04:42:56.038676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:10.325465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332224+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:42:55.332224+0000","last_peered":"2023-08-25T04:42:55.332224+0000","last_clean":"2023-08-25T04:42:55.332224+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:42:55.332224+0000","last_undegraded":"2023-08-25T04:42:55.332224+0000","last_fullsized":"2023-08-25T04:42:55.332224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:50:38.175235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499174+0000","last_change":"2023-08-25T04:42:50.329325+0000","last_active":"2023-08-25T04:42:55.499174+0000","last_peered":"2023-08-25T04:42:55.499174+0000","last_clean":"2023-08-25T04:42:55.499174+0000","last_became_active":"2023-08-25T04:42:50.329008+0000","last_became_peered":"2023-08-25T04:42:50.329008+0000","last_unstale":"2023-08-25T04:42:55.499174+0000","last_undegraded":"2023-08-25T04:42:55.499174+0000","last_fullsized":"2023-08-25T04:42:55.499174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:45.734751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038778+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:42:56.038778+0000","last_peered":"2023-08-25T04:42:56.038778+0000","last_clean":"2023-08-25T04:42:56.038778+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:42:56.038778+0000","last_undegraded":"2023-08-25T04:42:56.038778+0000","last_fullsized":"2023-08-25T04:42:56.038778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:50.076998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498529+0000","last_change":"2023-08-25T04:42:50.331990+0000","last_active":"2023-08-25T04:42:55.498529+0000","last_peered":"2023-08-25T04:42:55.498529+0000","last_clean":"2023-08-25T04:42:55.498529+0000","last_became_active":"2023-08-25T04:42:50.331614+0000","last_became_peered":"2023-08-25T04:42:50.331614+0000","last_unstale":"2023-08-25T04:42:55.498529+0000","last_undegraded":"2023-08-25T04:42:55.498529+0000","last_fullsized":"2023-08-25T04:42:55.498529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:48.466293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038263+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:42:56.038263+0000","last_peered":"2023-08-25T04:42:56.038263+0000","last_clean":"2023-08-25T04:42:56.038263+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:42:56.038263+0000","last_undegraded":"2023-08-25T04:42:56.038263+0000","last_fullsized":"2023-08-25T04:42:56.038263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:24.987336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332473+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:42:55.332473+0000","last_peered":"2023-08-25T04:42:55.332473+0000","last_clean":"2023-08-25T04:42:55.332473+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:42:55.332473+0000","last_undegraded":"2023-08-25T04:42:55.332473+0000","last_fullsized":"2023-08-25T04:42:55.332473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:47.251035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498372+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:42:55.498372+0000","last_peered":"2023-08-25T04:42:55.498372+0000","last_clean":"2023-08-25T04:42:55.498372+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:42:55.498372+0000","last_undegraded":"2023-08-25T04:42:55.498372+0000","last_fullsized":"2023-08-25T04:42:55.498372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:43:44.443815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499512+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:42:55.499512+0000","last_peered":"2023-08-25T04:42:55.499512+0000","last_clean":"2023-08-25T04:42:55.499512+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:42:55.499512+0000","last_undegraded":"2023-08-25T04:42:55.499512+0000","last_fullsized":"2023-08-25T04:42:55.499512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:53.310159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331924+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:42:55.331924+0000","last_peered":"2023-08-25T04:42:55.331924+0000","last_clean":"2023-08-25T04:42:55.331924+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:42:55.331924+0000","last_undegraded":"2023-08-25T04:42:55.331924+0000","last_fullsized":"2023-08-25T04:42:55.331924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:14.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499080+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:42:55.499080+0000","last_peered":"2023-08-25T04:42:55.499080+0000","last_clean":"2023-08-25T04:42:55.499080+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:42:55.499080+0000","last_undegraded":"2023-08-25T04:42:55.499080+0000","last_fullsized":"2023-08-25T04:42:55.499080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:03:40.257726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038256+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:42:56.038256+0000","last_peered":"2023-08-25T04:42:56.038256+0000","last_clean":"2023-08-25T04:42:56.038256+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:42:56.038256+0000","last_undegraded":"2023-08-25T04:42:56.038256+0000","last_fullsized":"2023-08-25T04:42:56.038256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:30:14.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332032+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:42:55.332032+0000","last_peered":"2023-08-25T04:42:55.332032+0000","last_clean":"2023-08-25T04:42:55.332032+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:42:55.332032+0000","last_undegraded":"2023-08-25T04:42:55.332032+0000","last_fullsized":"2023-08-25T04:42:55.332032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:36.560149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498804+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:42:55.498804+0000","last_peered":"2023-08-25T04:42:55.498804+0000","last_clean":"2023-08-25T04:42:55.498804+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:42:55.498804+0000","last_undegraded":"2023-08-25T04:42:55.498804+0000","last_fullsized":"2023-08-25T04:42:55.498804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:06.597833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332133+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:42:55.332133+0000","last_peered":"2023-08-25T04:42:55.332133+0000","last_clean":"2023-08-25T04:42:55.332133+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:42:55.332133+0000","last_undegraded":"2023-08-25T04:42:55.332133+0000","last_fullsized":"2023-08-25T04:42:55.332133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:32.544871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038780+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:42:56.038780+0000","last_peered":"2023-08-25T04:42:56.038780+0000","last_clean":"2023-08-25T04:42:56.038780+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:42:56.038780+0000","last_undegraded":"2023-08-25T04:42:56.038780+0000","last_fullsized":"2023-08-25T04:42:56.038780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:43:40.504423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497849+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:42:55.497849+0000","last_peered":"2023-08-25T04:42:55.497849+0000","last_clean":"2023-08-25T04:42:55.497849+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:42:55.497849+0000","last_undegraded":"2023-08-25T04:42:55.497849+0000","last_fullsized":"2023-08-25T04:42:55.497849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:51:52.545454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331405+0000","last_change":"2023-08-25T04:42:49.326537+0000","last_active":"2023-08-25T04:42:55.331405+0000","last_peered":"2023-08-25T04:42:55.331405+0000","last_clean":"2023-08-25T04:42:55.331405+0000","last_became_active":"2023-08-25T04:42:49.325361+0000","last_became_peered":"2023-08-25T04:42:49.325361+0000","last_unstale":"2023-08-25T04:42:55.331405+0000","last_undegraded":"2023-08-25T04:42:55.331405+0000","last_fullsized":"2023-08-25T04:42:55.331405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:49:55.273612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498837+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:42:55.498837+0000","last_peered":"2023-08-25T04:42:55.498837+0000","last_clean":"2023-08-25T04:42:55.498837+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:42:55.498837+0000","last_undegraded":"2023-08-25T04:42:55.498837+0000","last_fullsized":"2023-08-25T04:42:55.498837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:17.394975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499255+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:42:55.499255+0000","last_peered":"2023-08-25T04:42:55.499255+0000","last_clean":"2023-08-25T04:42:55.499255+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:42:55.499255+0000","last_undegraded":"2023-08-25T04:42:55.499255+0000","last_fullsized":"2023-08-25T04:42:55.499255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:03.796144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498750+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:42:55.498750+0000","last_peered":"2023-08-25T04:42:55.498750+0000","last_clean":"2023-08-25T04:42:55.498750+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:42:55.498750+0000","last_undegraded":"2023-08-25T04:42:55.498750+0000","last_fullsized":"2023-08-25T04:42:55.498750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:34.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498480+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:42:55.498480+0000","last_peered":"2023-08-25T04:42:55.498480+0000","last_clean":"2023-08-25T04:42:55.498480+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:42:55.498480+0000","last_undegraded":"2023-08-25T04:42:55.498480+0000","last_fullsized":"2023-08-25T04:42:55.498480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:22.278245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499059+0000","last_change":"2023-08-25T04:42:54.357813+0000","last_active":"2023-08-25T04:42:55.499059+0000","last_peered":"2023-08-25T04:42:55.499059+0000","last_clean":"2023-08-25T04:42:55.499059+0000","last_became_active":"2023-08-25T04:42:54.357400+0000","last_became_peered":"2023-08-25T04:42:54.357400+0000","last_unstale":"2023-08-25T04:42:55.499059+0000","last_undegraded":"2023-08-25T04:42:55.499059+0000","last_fullsized":"2023-08-25T04:42:55.499059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:35.674214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498104+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:42:55.498104+0000","last_peered":"2023-08-25T04:42:55.498104+0000","last_clean":"2023-08-25T04:42:55.498104+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:42:55.498104+0000","last_undegraded":"2023-08-25T04:42:55.498104+0000","last_fullsized":"2023-08-25T04:42:55.498104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:08.261819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331753+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:42:55.331753+0000","last_peered":"2023-08-25T04:42:55.331753+0000","last_clean":"2023-08-25T04:42:55.331753+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:42:55.331753+0000","last_undegraded":"2023-08-25T04:42:55.331753+0000","last_fullsized":"2023-08-25T04:42:55.331753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:45.227581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.331399+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:42:55.331399+0000","last_peered":"2023-08-25T04:42:55.331399+0000","last_clean":"2023-08-25T04:42:55.331399+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:42:55.331399+0000","last_undegraded":"2023-08-25T04:42:55.331399+0000","last_fullsized":"2023-08-25T04:42:55.331399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:22:21.252880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498044+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:42:55.498044+0000","last_peered":"2023-08-25T04:42:55.498044+0000","last_clean":"2023-08-25T04:42:55.498044+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:42:55.498044+0000","last_undegraded":"2023-08-25T04:42:55.498044+0000","last_fullsized":"2023-08-25T04:42:55.498044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:02.689977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497914+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:42:55.497914+0000","last_peered":"2023-08-25T04:42:55.497914+0000","last_clean":"2023-08-25T04:42:55.497914+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:42:55.497914+0000","last_undegraded":"2023-08-25T04:42:55.497914+0000","last_fullsized":"2023-08-25T04:42:55.497914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:22.085043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498363+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:42:55.498363+0000","last_peered":"2023-08-25T04:42:55.498363+0000","last_clean":"2023-08-25T04:42:55.498363+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:42:55.498363+0000","last_undegraded":"2023-08-25T04:42:55.498363+0000","last_fullsized":"2023-08-25T04:42:55.498363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:36:10.911830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498328+0000","last_change":"2023-08-25T04:42:49.327175+0000","last_active":"2023-08-25T04:42:55.498328+0000","last_peered":"2023-08-25T04:42:55.498328+0000","last_clean":"2023-08-25T04:42:55.498328+0000","last_became_active":"2023-08-25T04:42:49.326884+0000","last_became_peered":"2023-08-25T04:42:49.326884+0000","last_unstale":"2023-08-25T04:42:55.498328+0000","last_undegraded":"2023-08-25T04:42:55.498328+0000","last_fullsized":"2023-08-25T04:42:55.498328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:57:48.362984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498436+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:42:55.498436+0000","last_peered":"2023-08-25T04:42:55.498436+0000","last_clean":"2023-08-25T04:42:55.498436+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:42:55.498436+0000","last_undegraded":"2023-08-25T04:42:55.498436+0000","last_fullsized":"2023-08-25T04:42:55.498436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:32:00.600660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498507+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:42:55.498507+0000","last_peered":"2023-08-25T04:42:55.498507+0000","last_clean":"2023-08-25T04:42:55.498507+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:42:55.498507+0000","last_undegraded":"2023-08-25T04:42:55.498507+0000","last_fullsized":"2023-08-25T04:42:55.498507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:34:51.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332068+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:42:55.332068+0000","last_peered":"2023-08-25T04:42:55.332068+0000","last_clean":"2023-08-25T04:42:55.332068+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:42:55.332068+0000","last_undegraded":"2023-08-25T04:42:55.332068+0000","last_fullsized":"2023-08-25T04:42:55.332068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:32:33.439250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332423+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:42:55.332423+0000","last_peered":"2023-08-25T04:42:55.332423+0000","last_clean":"2023-08-25T04:42:55.332423+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:42:55.332423+0000","last_undegraded":"2023-08-25T04:42:55.332423+0000","last_fullsized":"2023-08-25T04:42:55.332423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:18:35.106901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498795+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:42:55.498795+0000","last_peered":"2023-08-25T04:42:55.498795+0000","last_clean":"2023-08-25T04:42:55.498795+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:42:55.498795+0000","last_undegraded":"2023-08-25T04:42:55.498795+0000","last_fullsized":"2023-08-25T04:42:55.498795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:29.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498789+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:42:55.498789+0000","last_peered":"2023-08-25T04:42:55.498789+0000","last_clean":"2023-08-25T04:42:55.498789+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:42:55.498789+0000","last_undegraded":"2023-08-25T04:42:55.498789+0000","last_fullsized":"2023-08-25T04:42:55.498789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:30.087944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332775+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:42:55.332775+0000","last_peered":"2023-08-25T04:42:55.332775+0000","last_clean":"2023-08-25T04:42:55.332775+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:42:55.332775+0000","last_undegraded":"2023-08-25T04:42:55.332775+0000","last_fullsized":"2023-08-25T04:42:55.332775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:40:22.161009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497983+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:42:55.497983+0000","last_peered":"2023-08-25T04:42:55.497983+0000","last_clean":"2023-08-25T04:42:55.497983+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:42:55.497983+0000","last_undegraded":"2023-08-25T04:42:55.497983+0000","last_fullsized":"2023-08-25T04:42:55.497983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:17:20.821857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.500462+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:42:55.500462+0000","last_peered":"2023-08-25T04:42:55.500462+0000","last_clean":"2023-08-25T04:42:55.500462+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:42:55.500462+0000","last_undegraded":"2023-08-25T04:42:55.500462+0000","last_fullsized":"2023-08-25T04:42:55.500462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:17.667227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332398+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:42:55.332398+0000","last_peered":"2023-08-25T04:42:55.332398+0000","last_clean":"2023-08-25T04:42:55.332398+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:42:55.332398+0000","last_undegraded":"2023-08-25T04:42:55.332398+0000","last_fullsized":"2023-08-25T04:42:55.332398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:21:30.344818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.497999+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:42:55.497999+0000","last_peered":"2023-08-25T04:42:55.497999+0000","last_clean":"2023-08-25T04:42:55.497999+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:42:55.497999+0000","last_undegraded":"2023-08-25T04:42:55.497999+0000","last_fullsized":"2023-08-25T04:42:55.497999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:07:58.375742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038864+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:42:56.038864+0000","last_peered":"2023-08-25T04:42:56.038864+0000","last_clean":"2023-08-25T04:42:56.038864+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:42:56.038864+0000","last_undegraded":"2023-08-25T04:42:56.038864+0000","last_fullsized":"2023-08-25T04:42:56.038864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:18:13.540910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.332682+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:42:55.332682+0000","last_peered":"2023-08-25T04:42:55.332682+0000","last_clean":"2023-08-25T04:42:55.332682+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:42:55.332682+0000","last_undegraded":"2023-08-25T04:42:55.332682+0000","last_fullsized":"2023-08-25T04:42:55.332682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:42.788095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498068+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:42:55.498068+0000","last_peered":"2023-08-25T04:42:55.498068+0000","last_clean":"2023-08-25T04:42:55.498068+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:42:55.498068+0000","last_undegraded":"2023-08-25T04:42:55.498068+0000","last_fullsized":"2023-08-25T04:42:55.498068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:39.945056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499361+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:42:55.499361+0000","last_peered":"2023-08-25T04:42:55.499361+0000","last_clean":"2023-08-25T04:42:55.499361+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:42:55.499361+0000","last_undegraded":"2023-08-25T04:42:55.499361+0000","last_fullsized":"2023-08-25T04:42:55.499361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:28:28.952965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498165+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:42:55.498165+0000","last_peered":"2023-08-25T04:42:55.498165+0000","last_clean":"2023-08-25T04:42:55.498165+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:42:55.498165+0000","last_undegraded":"2023-08-25T04:42:55.498165+0000","last_fullsized":"2023-08-25T04:42:55.498165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:05.052741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498192+0000","last_change":"2023-08-25T04:42:50.330514+0000","last_active":"2023-08-25T04:42:55.498192+0000","last_peered":"2023-08-25T04:42:55.498192+0000","last_clean":"2023-08-25T04:42:55.498192+0000","last_became_active":"2023-08-25T04:42:50.330001+0000","last_became_peered":"2023-08-25T04:42:50.330001+0000","last_unstale":"2023-08-25T04:42:55.498192+0000","last_undegraded":"2023-08-25T04:42:55.498192+0000","last_fullsized":"2023-08-25T04:42:55.498192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:21.145109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038823+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:42:56.038823+0000","last_peered":"2023-08-25T04:42:56.038823+0000","last_clean":"2023-08-25T04:42:56.038823+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:42:56.038823+0000","last_undegraded":"2023-08-25T04:42:56.038823+0000","last_fullsized":"2023-08-25T04:42:56.038823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:35:11.398547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499668+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:42:55.499668+0000","last_peered":"2023-08-25T04:42:55.499668+0000","last_clean":"2023-08-25T04:42:55.499668+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:42:55.499668+0000","last_undegraded":"2023-08-25T04:42:55.499668+0000","last_fullsized":"2023-08-25T04:42:55.499668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:32.084873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499048+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:42:55.499048+0000","last_peered":"2023-08-25T04:42:55.499048+0000","last_clean":"2023-08-25T04:42:55.499048+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:42:55.499048+0000","last_undegraded":"2023-08-25T04:42:55.499048+0000","last_fullsized":"2023-08-25T04:42:55.499048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:05:26.906447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498927+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:42:55.498927+0000","last_peered":"2023-08-25T04:42:55.498927+0000","last_clean":"2023-08-25T04:42:55.498927+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:42:55.498927+0000","last_undegraded":"2023-08-25T04:42:55.498927+0000","last_fullsized":"2023-08-25T04:42:55.498927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:46.733202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498390+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:42:55.498390+0000","last_peered":"2023-08-25T04:42:55.498390+0000","last_clean":"2023-08-25T04:42:55.498390+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:42:55.498390+0000","last_undegraded":"2023-08-25T04:42:55.498390+0000","last_fullsized":"2023-08-25T04:42:55.498390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:50:23.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498165+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:42:55.498165+0000","last_peered":"2023-08-25T04:42:55.498165+0000","last_clean":"2023-08-25T04:42:55.498165+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:42:55.498165+0000","last_undegraded":"2023-08-25T04:42:55.498165+0000","last_fullsized":"2023-08-25T04:42:55.498165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:26.391472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.499133+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:42:55.499133+0000","last_peered":"2023-08-25T04:42:55.499133+0000","last_clean":"2023-08-25T04:42:55.499133+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:42:55.499133+0000","last_undegraded":"2023-08-25T04:42:55.499133+0000","last_fullsized":"2023-08-25T04:42:55.499133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:09:50.780046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:56.038634+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:42:56.038634+0000","last_peered":"2023-08-25T04:42:56.038634+0000","last_clean":"2023-08-25T04:42:56.038634+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:42:56.038634+0000","last_undegraded":"2023-08-25T04:42:56.038634+0000","last_fullsized":"2023-08-25T04:42:56.038634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:33.955508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-25T04:42:55.498727+0000","last_change":"2023-08-25T04:42:50.327235+0000","last_active":"2023-08-25T04:42:55.498727+0000","last_peered":"2023-08-25T04:42:55.498727+0000","last_clean":"2023-08-25T04:42:55.498727+0000","last_became_active":"2023-08-25T04:42:50.326486+0000","last_became_peered":"2023-08-25T04:42:50.326486+0000","last_unstale":"2023-08-25T04:42:55.498727+0000","last_undegraded":"2023-08-25T04:42:55.498727+0000","last_fullsized":"2023-08-25T04:42:55.498727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:26.595739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":15308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":463755,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79300000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":9,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.0249999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.90200000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1058159,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.443},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.629},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.80900000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78400000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":464337,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.62}]},{"osd":1,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.505},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.748}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103197,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":464028,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.622}]},{"osd":1,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.29599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.498}]},{"osd":3,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77800000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:42:26 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.98199999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.87},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1054374,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.42099999999999999,"15min":0.40999999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.41899999999999998,"15min":0.40500000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86799999999999999,"5min":1.026,"15min":1.026},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.53500000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":0.996,"15min":0.996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93200000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58599999999999997}]},{"osd":9,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60799999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":463755,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.032,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70199999999999996,"15min":0.69699999999999995},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.119,"5min":1.131,"15min":1.131},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.68600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.68300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.70899999999999996}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.177,"5min":1.389,"15min":1.389},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.752},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.874},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":467324,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83199999999999996,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85299999999999998,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.63}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54800000000000004,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.54200000000000004,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.153,"5min":1.548,"15min":1.548},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.441,"15min":0.41599999999999998},"min":{"1min":0.33200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.503}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":1.613,"15min":1.613},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89700000000000002,"5min":1.597,"15min":1.597},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Aug 25 04:42:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92000000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:42:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":465233,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.44600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.373,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.77100000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.56899999999999995},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":1.194,"15min":1.194},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81699999999999995,"5min":1.488,"15min":1.488},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.99399999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94299999999999995,"5min":1.603,"15min":1.603},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1930000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1599999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.97999999999999998,"5min":1.534,"15min":1.534},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.751,"15min":0.755},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92000000000000004,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.96599999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.056,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0840000000000001,"5min":1.365,"15min":1.365},"last":0.874}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.002}]},{"osd":10,"last update":"Fri Aug 25 04:42:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri Aug 25 04:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.01},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.94199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":467631,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76000000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63}]},{"osd":1,"last update":"Fri Aug 25 04:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.53100000000000003,"15min":0.501},"min":{"1min":0.40600000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Aug 25 04:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.879,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:42:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.878},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Aug 25 04:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81899999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.0569999999999999}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.92000000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.071},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":1.04}]},{"osd":10,"last update":"Fri Aug 25 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.091},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":1.0249999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":463859,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.438,"15min":0.41699999999999998},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.505,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.004,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.751},{"interface":"front","average":{"1min":0.748,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.57099999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":1.53,"15min":1.53},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":1.573,"15min":1.573},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Aug 25 04:42:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.621},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2370000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92500000000000004,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85299999999999998,"5min":1.556,"15min":1.556},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.98199999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.73499999999999999,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.69099999999999995}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.879}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920814,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463777,"data_compressed":11697,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.36599999999999999,"15min":0.34599999999999997},"min":{"1min":0.30299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.378,"15min":0.36099999999999999},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84999999999999998,"5min":0.876,"15min":0.876},"last":0.499}]},{"osd":3,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.432},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69599999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93400000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Aug 25 04:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.59899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1054305,"data_compressed":47370,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.53600000000000003,"15min":0.5},"min":{"1min":0.432,"5min":0.318,"15min":0.318},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.30299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.371},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.39400000000000002},"min":{"1min":0.30499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.624},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:42:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.503},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57199999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.749},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:42:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:43:18.455 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T04:43:18.456 INFO:tasks.cephadm:Setting up client nodes... 2023-08-25T04:43:18.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-08-25T04:43:19.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:18 smithi145 ceph-mon[97681]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:18 smithi145 ceph-mon[97681]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:18 smithi162 ceph-mon[130941]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:18 smithi162 ceph-mon[130941]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:18 smithi183 ceph-mon[131414]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-25T04:43:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:18 smithi183 ceph-mon[131414]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:19.206 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:20.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:19 smithi145 ceph-mon[97681]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:19 smithi162 ceph-mon[130941]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:19 smithi183 ceph-mon[131414]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:43:20.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 -- 172.21.15.145:0/2205397883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc030101be0 msgr2=0x7fc030102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:20.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 --2- 172.21.15.145:0/2205397883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc030101be0 0x7fc030102040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc020009920 tx=0x7fc020009c30 comp rx=0 tx=0).stop 2023-08-25T04:43:20.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 -- 172.21.15.145:0/2205397883 shutdown_connections 2023-08-25T04:43:20.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 --2- 172.21.15.145:0/2205397883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030102580 0x7fc030108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 --2- 172.21.15.145:0/2205397883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0301009e0 0x7fc030100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.619+0000 7fc03746a700 1 --2- 172.21.15.145:0/2205397883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc030101be0 0x7fc030102040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.620+0000 7fc03746a700 1 -- 172.21.15.145:0/2205397883 >> 172.21.15.145:0/2205397883 conn(0x7fc0300fbfa0 msgr2=0x7fc0300fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:20.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.620+0000 7fc03746a700 1 -- 172.21.15.145:0/2205397883 shutdown_connections 2023-08-25T04:43:20.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.620+0000 7fc03746a700 1 -- 172.21.15.145:0/2205397883 wait complete. 2023-08-25T04:43:20.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 Processor -- start 2023-08-25T04:43:20.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 -- start start 2023-08-25T04:43:20.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:20.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 0x7fc030072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:20.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc030102580 0x7fc030073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:20.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300739d0 con 0x7fc0301009e0 2023-08-25T04:43:20.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030073b40 con 0x7fc030101be0 2023-08-25T04:43:20.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc03746a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0301a07d0 con 0x7fc030102580 2023-08-25T04:43:20.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.621+0000 7fc035206700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:20.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56702/0 (socket says 172.21.15.145:56702) 2023-08-25T04:43:20.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 -- 172.21.15.145:0/1424258269 learned_addr learned my addr 172.21.15.145:0/1424258269 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:20.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035a07700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc030102580 0x7fc030073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:20.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc034a05700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 0x7fc030072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 -- 172.21.15.145:0/1424258269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc030102580 msgr2=0x7fc030073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc030102580 0x7fc030073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 -- 172.21.15.145:0/1424258269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 msgr2=0x7fc030072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 0x7fc030072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 -- 172.21.15.145:0/1424258269 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009520 con 0x7fc0301009e0 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc034a05700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 0x7fc030072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:20.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc035206700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9f0 tx=0x7fc02c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004220 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc02c004380 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301a0ab0 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.622+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c003be0 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.623+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301a1000 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.624+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03010c230 con 0x7fc0301009e0 2023-08-25T04:43:20.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.626+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc02c003d40 con 0x7fc0301009e0 2023-08-25T04:43:20.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.627+0000 7fc0267fc700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 0x7fc01c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:20.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.627+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc02c014070 con 0x7fc0301009e0 2023-08-25T04:43:20.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.627+0000 7fc034a05700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 0x7fc01c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:20.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.628+0000 7fc034a05700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 0x7fc01c0a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc02000fa40 tx=0x7fc020019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:20.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.631+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc02c094fd0 con 0x7fc0301009e0 2023-08-25T04:43:20.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.909+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc0301079d0 con 0x7fc0301009e0 2023-08-25T04:43:20.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.914+0000 7fc0267fc700 1 -- 172.21.15.145:0/1424258269 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc02c0985f0 con 0x7fc0301009e0 2023-08-25T04:43:20.916 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2023-08-25T04:43:20.916 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBoMehkUD5ZNhAA7g9UhxbZNIMl9eutgEYvLw== 2023-08-25T04:43:20.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 msgr2=0x7fc01c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:20.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 0x7fc01c0a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc02000fa40 tx=0x7fc020019040 comp rx=0 tx=0).stop 2023-08-25T04:43:20.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 msgr2=0x7fc0300746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:20.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9f0 tx=0x7fc02c00ed00 comp rx=0 tx=0).stop 2023-08-25T04:43:20.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 shutdown_connections 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc01c09ffd0 0x7fc01c0a2490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc030102580 0x7fc030073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0301009e0 0x7fc0300746c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 --2- 172.21.15.145:0/1424258269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc030101be0 0x7fc030072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.916+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 >> 172.21.15.145:0/1424258269 conn(0x7fc0300fbfa0 msgr2=0x7fc0300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.917+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 shutdown_connections 2023-08-25T04:43:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:20.917+0000 7fc03746a700 1 -- 172.21.15.145:0/1424258269 wait complete. 2023-08-25T04:43:21.044 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:20 smithi145 ceph-mon[97681]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-08-25T04:43:21.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:20 smithi162 ceph-mon[130941]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-08-25T04:43:21.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:20 smithi183 ceph-mon[131414]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-08-25T04:43:21.306 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:43:21.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-08-25T04:43:21.306 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-25T04:43:21.408 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-25T04:43:21.409 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-08-25T04:43:21.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph mgr dump --format=json 2023-08-25T04:43:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T04:43:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T04:43:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T04:43:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T04:43:22.169 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-25T04:43:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1424258269' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-25T04:43:23.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:22 smithi145 ceph-mon[97681]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-08-25T04:43:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:22 smithi183 ceph-mon[131414]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-08-25T04:43:23.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:22 smithi162 ceph-mon[130941]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-08-25T04:43:23.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 -- 172.21.15.145:0/3385690165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c300688e0 msgr2=0x7f8c30068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:23.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 --2- 172.21.15.145:0/3385690165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c300688e0 0x7f8c30068cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c200099c0 tx=0x7f8c20009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 -- 172.21.15.145:0/3385690165 shutdown_connections 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 --2- 172.21.15.145:0/3385690165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105d50 0x7f8c3010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 --2- 172.21.15.145:0/3385690165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c300688e0 0x7f8c30068cc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 --2- 172.21.15.145:0/3385690165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c30105390 0x7f8c30105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 -- 172.21.15.145:0/3385690165 >> 172.21.15.145:0/3385690165 conn(0x7f8c30076920 msgr2=0x7f8c30076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 -- 172.21.15.145:0/3385690165 shutdown_connections 2023-08-25T04:43:23.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.585+0000 7f8c36097700 1 -- 172.21.15.145:0/3385690165 wait complete. 2023-08-25T04:43:23.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.586+0000 7f8c36097700 1 Processor -- start 2023-08-25T04:43:23.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.586+0000 7f8c36097700 1 -- start start 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105390 0x7f8c3019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301990e0 con 0x7f8c30105d50 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30199250 con 0x7f8c300688e0 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c36097700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30199410 con 0x7f8c30105390 2023-08-25T04:43:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105390 0x7f8c3019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54102/0 (socket says 172.21.15.145:54102) 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57610/0 (socket says 172.21.15.145:57610) 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.587+0000 7f8c2f7fe700 1 -- 172.21.15.145:0/3052276465 learned_addr learned my addr 172.21.15.145:0/3052276465 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 -- 172.21.15.145:0/3052276465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105390 msgr2=0x7f8c3019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:23.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105390 0x7f8c3019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 -- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 msgr2=0x7f8c30198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:23.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 -- 172.21.15.145:0/3052276465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200095f0 con 0x7f8c300688e0 2023-08-25T04:43:23.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2ffff700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:23.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2f7fe700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000f690 tx=0x7f8c2000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:23.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2001d070 con 0x7f8c300688e0 2023-08-25T04:43:23.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c20005050 con 0x7f8c300688e0 2023-08-25T04:43:23.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c30199690 con 0x7f8c300688e0 2023-08-25T04:43:23.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20004970 con 0x7f8c300688e0 2023-08-25T04:43:23.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.588+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c301a59e0 con 0x7f8c300688e0 2023-08-25T04:43:23.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.589+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3004e620 con 0x7f8c300688e0 2023-08-25T04:43:23.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.591+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f8c200051c0 con 0x7f8c300688e0 2023-08-25T04:43:23.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.593+0000 7f8c2cff9700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 0x7f8c180a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:23.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.593+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8c200cc600 con 0x7f8c300688e0 2023-08-25T04:43:23.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.593+0000 7f8c2effd700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 0x7f8c180a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:23.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.593+0000 7f8c2effd700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 0x7f8c180a2490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c30106cf0 tx=0x7f8c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:23.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.595+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8c200961d0 con 0x7f8c300688e0 2023-08-25T04:43:23.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.851+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8c301108f0 con 0x7f8c300688e0 2023-08-25T04:43:23.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.857+0000 7f8c2cff9700 1 -- 172.21.15.145:0/3052276465 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+258017 (secure 0 0 0) 0x7f8c200997f0 con 0x7f8c300688e0 2023-08-25T04:43:23.860 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:23.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 msgr2=0x7f8c180a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 0x7f8c180a2490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c30106cf0 tx=0x7f8c24009260 comp rx=0 tx=0).stop 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 msgr2=0x7f8c3019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000f690 tx=0x7f8c2000bfd0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 shutdown_connections 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8c1809ffd0 0x7f8c180a2490 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c30105d50 0x7f8c30198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300688e0 0x7f8c3019e9f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 --2- 172.21.15.145:0/3052276465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c30105390 0x7f8c3019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:23.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 >> 172.21.15.145:0/3052276465 conn(0x7f8c30076920 msgr2=0x7f8c3010ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:23.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 shutdown_connections 2023-08-25T04:43:23.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:23.861+0000 7f8c36097700 1 -- 172.21.15.145:0/3052276465 wait complete. 2023-08-25T04:43:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:23 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3052276465' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T04:43:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:23 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3052276465' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T04:43:24.364 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":16,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":891678096},{"type":"v1","addr":"172.21.15.145:6801","nonce":891678096}]},"active_addr":"172.21.15.145:6801/891678096","active_change":"2023-08-25T04:36:33.606486+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.145:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":4072524950}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":3264758001}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":4132549840}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":459521375}]}]} 2023-08-25T04:43:24.375 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-08-25T04:43:24.375 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-08-25T04:43:24.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T04:43:24.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:23 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3052276465' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-25T04:43:25.128 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:24 smithi145 ceph-mon[97681]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-25T04:43:25.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:24 smithi162 ceph-mon[130941]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-25T04:43:25.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:24 smithi183 ceph-mon[131414]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-25T04:43:26.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 -- 172.21.15.145:0/3072150931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c100940 msgr2=0x7f3a2c100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:26.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 --2- 172.21.15.145:0/3072150931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c100940 0x7f3a2c100dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009990 tx=0x7f3a1c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 -- 172.21.15.145:0/3072150931 shutdown_connections 2023-08-25T04:43:26.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 --2- 172.21.15.145:0/3072150931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c101300 0x7f3a2c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 --2- 172.21.15.145:0/3072150931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c106940 0x7f3a2c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 --2- 172.21.15.145:0/3072150931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c100940 0x7f3a2c100dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.517+0000 7f3a333fc700 1 -- 172.21.15.145:0/3072150931 >> 172.21.15.145:0/3072150931 conn(0x7f3a2c0fc410 msgr2=0x7f3a2c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:26.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.518+0000 7f3a333fc700 1 -- 172.21.15.145:0/3072150931 shutdown_connections 2023-08-25T04:43:26.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.518+0000 7f3a333fc700 1 -- 172.21.15.145:0/3072150931 wait complete. 2023-08-25T04:43:26.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 Processor -- start 2023-08-25T04:43:26.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 -- start start 2023-08-25T04:43:26.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:26.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:26.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c106940 0x7f3a2c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:26.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c199140 con 0x7f3a2c101300 2023-08-25T04:43:26.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c1992b0 con 0x7f3a2c106940 2023-08-25T04:43:26.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.519+0000 7f3a333fc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c199470 con 0x7f3a2c100940 2023-08-25T04:43:26.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:26.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a31198700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:26.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a31198700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48746/0 (socket says 172.21.15.145:48746) 2023-08-25T04:43:26.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57620/0 (socket says 172.21.15.145:57620) 2023-08-25T04:43:26.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a31198700 1 -- 172.21.15.145:0/2206764952 learned_addr learned my addr 172.21.15.145:0/2206764952 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:26.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a31999700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c106940 0x7f3a2c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:26.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 -- 172.21.15.145:0/2206764952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 msgr2=0x7f3a2c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:26.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 -- 172.21.15.145:0/2206764952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c106940 msgr2=0x7f3a2c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:26.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c106940 0x7f3a2c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 -- 172.21.15.145:0/2206764952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0095f0 con 0x7f3a2c101300 2023-08-25T04:43:26.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a31198700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:26.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.520+0000 7f3a30997700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0098e0 tx=0x7f3a1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:26.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.521+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c01d070 con 0x7f3a2c101300 2023-08-25T04:43:26.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.521+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c005030 con 0x7f3a2c101300 2023-08-25T04:43:26.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.521+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c1996f0 con 0x7f3a2c101300 2023-08-25T04:43:26.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.521+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004ab0 con 0x7f3a2c101300 2023-08-25T04:43:26.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.521+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c1a5a40 con 0x7f3a2c101300 2023-08-25T04:43:26.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.522+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c04e620 con 0x7f3a2c101300 2023-08-25T04:43:26.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.524+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3a1c004cf0 con 0x7f3a2c101300 2023-08-25T04:43:26.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.525+0000 7f3a227fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 0x7f3a180a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:26.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.525+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a1c0cc490 con 0x7f3a2c101300 2023-08-25T04:43:26.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.525+0000 7f3a31198700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 0x7f3a180a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:26.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.526+0000 7f3a31198700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 0x7f3a180a2490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011550 tx=0x7f3a28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:26.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.530+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3a1c096060 con 0x7f3a2c101300 2023-08-25T04:43:26.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.728+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a2c110a10 con 0x7f3a2c101300 2023-08-25T04:43:26.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.730+0000 7f3a227fc700 1 -- 172.21.15.145:0/2206764952 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7f3a1c099680 con 0x7f3a2c101300 2023-08-25T04:43:26.731 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:26.731 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:26.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 msgr2=0x7f3a180a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:26.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 0x7f3a180a2490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011550 tx=0x7f3a28009260 comp rx=0 tx=0).stop 2023-08-25T04:43:26.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 msgr2=0x7f3a2c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0098e0 tx=0x7f3a1c004730 comp rx=0 tx=0).stop 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 shutdown_connections 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3a1809ffd0 0x7f3a180a2490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a2c106940 0x7f3a2c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a2c100940 0x7f3a2c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 --2- 172.21.15.145:0/2206764952 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a2c101300 0x7f3a2c19ef90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:26.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 >> 172.21.15.145:0/2206764952 conn(0x7f3a2c0fc410 msgr2=0x7f3a2c10bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:26.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 shutdown_connections 2023-08-25T04:43:26.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:26.733+0000 7f3a333fc700 1 -- 172.21.15.145:0/2206764952 wait complete. 2023-08-25T04:43:27.063 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:26 smithi145 ceph-mon[97681]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T04:43:27.063 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2206764952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:27.117 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-08-25T04:43:27.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd dump --format=json 2023-08-25T04:43:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:26 smithi162 ceph-mon[130941]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T04:43:27.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:26 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2206764952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:26 smithi183 ceph-mon[131414]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-08-25T04:43:27.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:26 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2206764952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:27.838 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:29.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:28 smithi145 ceph-mon[97681]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-08-25T04:43:29.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 -- 172.21.15.145:0/1165551274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec106940 msgr2=0x7fccec106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:29.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 --2- 172.21.15.145:0/1165551274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec106940 0x7fccec106d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009990 tx=0x7fcce8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 -- 172.21.15.145:0/1165551274 shutdown_connections 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 --2- 172.21.15.145:0/1165551274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 0x7fccec10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 --2- 172.21.15.145:0/1165551274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec106940 0x7fccec106d20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 --2- 172.21.15.145:0/1165551274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec100940 0x7fccec100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 -- 172.21.15.145:0/1165551274 >> 172.21.15.145:0/1165551274 conn(0x7fccec0fc410 msgr2=0x7fccec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 -- 172.21.15.145:0/1165551274 shutdown_connections 2023-08-25T04:43:29.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.352+0000 7fccf3336700 1 -- 172.21.15.145:0/1165551274 wait complete. 2023-08-25T04:43:29.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.353+0000 7fccf3336700 1 Processor -- start 2023-08-25T04:43:29.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.353+0000 7fccf3336700 1 -- start start 2023-08-25T04:43:29.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:29.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 0x7fccec19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106940 0x7fccec198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1992e0 con 0x7fccec100940 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec199450 con 0x7fccec101300 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf3336700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec199610 con 0x7fccec106940 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:29.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf08d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 0x7fccec19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57638/0 (socket says 172.21.15.145:57638) 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 -- 172.21.15.145:0/3037381146 learned_addr learned my addr 172.21.15.145:0/3037381146 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf18d3700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106940 0x7fccec198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 -- 172.21.15.145:0/3037381146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106940 msgr2=0x7fccec198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106940 0x7fccec198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 -- 172.21.15.145:0/3037381146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 msgr2=0x7fccec19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 0x7fccec19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.354+0000 7fccf10d2700 1 -- 172.21.15.145:0/3037381146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80095f0 con 0x7fccec100940 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fccf10d2700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcce8005910 tx=0x7fcce8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce801d070 con 0x7fccec100940 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce8022470 con 0x7fccec100940 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8017650 con 0x7fccec100940 2023-08-25T04:43:29.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec199890 con 0x7fccec100940 2023-08-25T04:43:29.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.355+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1a5ac0 con 0x7fccec100940 2023-08-25T04:43:29.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.356+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec199a50 con 0x7fccec100940 2023-08-25T04:43:29.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.358+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcce80225e0 con 0x7fccec100940 2023-08-25T04:43:29.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.359+0000 7fcce27fc700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 0x7fccd80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:29.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.360+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcce80cc810 con 0x7fccec100940 2023-08-25T04:43:29.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.360+0000 7fccf08d1700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 0x7fccd80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:29.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.360+0000 7fccf08d1700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 0x7fccd80a2410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fccec19a360 tx=0x7fccdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:29.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.361+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcce80963e0 con 0x7fccec100940 2023-08-25T04:43:29.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:28 smithi162 ceph-mon[130941]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-08-25T04:43:29.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:28 smithi183 ceph-mon[131414]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-08-25T04:43:29.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.560+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccec199a20 con 0x7fccec100940 2023-08-25T04:43:29.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.562+0000 7fcce27fc700 1 -- 172.21.15.145:0/3037381146 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24006 (secure 0 0 0) 0x7fcce8099a00 con 0x7fccec100940 2023-08-25T04:43:29.563 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":87,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:42:54.843584+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:43:29.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 msgr2=0x7fccd80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:29.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 0x7fccd80a2410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fccec19a360 tx=0x7fccdc009260 comp rx=0 tx=0).stop 2023-08-25T04:43:29.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 msgr2=0x7fccec19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:29.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcce8005910 tx=0x7fcce8004710 comp rx=0 tx=0).stop 2023-08-25T04:43:29.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 shutdown_connections 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fccd809ff50 0x7fccd80a2410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106940 0x7fccec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccec100940 0x7fccec19eb40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 --2- 172.21.15.145:0/3037381146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccec101300 0x7fccec19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 >> 172.21.15.145:0/3037381146 conn(0x7fccec0fc410 msgr2=0x7fccec10bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 shutdown_connections 2023-08-25T04:43:29.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:29.564+0000 7fccf3336700 1 -- 172.21.15.145:0/3037381146 wait complete. 2023-08-25T04:43:29.940 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-08-25T04:43:29.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-08-25T04:43:29.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-08-25T04:43:29.943 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-08-25T04:43:29.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-08-25T04:43:29.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-08-25T04:43:29.945 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-08-25T04:43:29.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-08-25T04:43:29.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-08-25T04:43:29.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-08-25T04:43:29.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-08-25T04:43:29.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-08-25T04:43:30.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:30 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3037381146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:30 smithi162 ceph-mon[130941]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-08-25T04:43:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:30 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3037381146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:30 smithi183 ceph-mon[131414]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-08-25T04:43:31.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:30 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3037381146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:43:31.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:30 smithi145 ceph-mon[97681]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-08-25T04:43:31.627 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.058 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.139 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.140 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.143 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.145 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.146 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.181 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.271 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.316 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.317 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.339 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:32.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:32 smithi145 ceph-mon[97681]: pgmap v286: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-08-25T04:43:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:32 smithi162 ceph-mon[130941]: pgmap v286: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-08-25T04:43:33.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:32 smithi183 ceph-mon[131414]: pgmap v286: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-08-25T04:43:34.055 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-08-25T04:43:34.056 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-08-25T04:43:34.056 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-08-25T04:43:34.056 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.056 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-08-25T04:43:34.056 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.057 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]: dispatch 2023-08-25T04:43:34.057 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-08-25T04:43:34.057 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:33 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-25T04:43:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-08-25T04:43:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-08-25T04:43:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-08-25T04:43:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-08-25T04:43:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]: dispatch 2023-08-25T04:43:34.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-08-25T04:43:34.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:33 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-25T04:43:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-08-25T04:43:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]: dispatch 2023-08-25T04:43:34.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-08-25T04:43:34.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:33 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-08-25T04:43:35.035 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-08-25T04:43:35.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-08-25T04:43:35.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]': finished 2023-08-25T04:43:35.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-08-25T04:43:35.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-25T04:43:35.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:34 smithi145 ceph-mon[97681]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T04:43:35.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-08-25T04:43:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-08-25T04:43:35.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]': finished 2023-08-25T04:43:35.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-08-25T04:43:35.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-25T04:43:35.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:34 smithi162 ceph-mon[130941]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T04:43:35.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-08-25T04:43:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11, 5, 8]}]': finished 2023-08-25T04:43:35.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-08-25T04:43:35.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-08-25T04:43:35.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:34 smithi183 ceph-mon[131414]: osdmap e88: 12 total, 12 up, 12 in 2023-08-25T04:43:35.929 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:35 smithi145 ceph-mon[97681]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T04:43:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:35 smithi162 ceph-mon[130941]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T04:43:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:35 smithi183 ceph-mon[131414]: osdmap e89: 12 total, 12 up, 12 in 2023-08-25T04:43:36.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 -- 172.21.15.145:0/917147260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c102db0 msgr2=0x7f960c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 --2- 172.21.15.145:0/917147260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c102db0 0x7f960c103190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9600009990 tx=0x7f9600009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 -- 172.21.15.145:0/917147260 shutdown_connections 2023-08-25T04:43:36.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 --2- 172.21.15.145:0/917147260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c107e90 0x7f960c10fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 --2- 172.21.15.145:0/917147260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c102db0 0x7f960c103190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 --2- 172.21.15.145:0/917147260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 0x7f960c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 -- 172.21.15.145:0/917147260 >> 172.21.15.145:0/917147260 conn(0x7f960c0fe620 msgr2=0x7f960c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 -- 172.21.15.145:0/917147260 shutdown_connections 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.356+0000 7f9612c35700 1 -- 172.21.15.145:0/917147260 wait complete. 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.358+0000 7f9612c35700 1 Processor -- start 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.358+0000 7f9612c35700 1 -- start start 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 0x7f960c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:36.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 0x7f960c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c107e90 0x7f960c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1990e0 con 0x7f960c107e90 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c199250 con 0x7f960c102db0 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f9612c35700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c199410 con 0x7f960c103760 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 0x7f960c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f960bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 0x7f960c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96111d2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c107e90 0x7f960c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:36.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f960bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 0x7f960c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56874/0 (socket says 172.21.15.145:56874) 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 0x7f960c19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:38350/0 (socket says 172.21.15.145:38350) 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f960bfff700 1 -- 172.21.15.145:0/2297212061 learned_addr learned my addr 172.21.15.145:0/2297212061 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 -- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 msgr2=0x7f960c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c103760 0x7f960c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 -- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c107e90 msgr2=0x7f960c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c107e90 0x7f960c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96109d1700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000095f0 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.359+0000 7f96111d2700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f960c107e90 0x7f960c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f96109d1700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 0x7f960c19e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96000059d0 tx=0x7f9600005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960001d070 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9600022470 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c199690 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600017650 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.360+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c1a59e0 con 0x7f960c102db0 2023-08-25T04:43:36.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.361+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f960c04e620 con 0x7f960c102db0 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.363+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9600022890 con 0x7f960c102db0 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.364+0000 7f9609ffb700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f95fc09ff50 0x7f95fc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f96000cd1d0 con 0x7f960c102db0 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f960bfff700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f95fc09ff50 0x7f95fc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f9609ffb700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] conn(0x7f95fc0aaef0 0x7f95fc0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95fc0ada60 con 0x7f95fc0aaef0 2023-08-25T04:43:36.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96000cd540 con 0x7f960c102db0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f960bfff700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f95fc09ff50 0x7f95fc0a2410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ffca0 tx=0x7f95f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.365+0000 7f96111d2700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] conn(0x7f95fc0aaef0 0x7f95fc0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.366+0000 7f96111d2700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] conn(0x7f95fc0aaef0 0x7f95fc0ad390 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.367+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== osd.2 v2:172.21.15.183:6800/3244881082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f95fc0ada60 con 0x7f95fc0aaef0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.385+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 --> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f960c107140 con 0x7f95fc0aaef0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9609ffb700 1 -- 172.21.15.145:0/2297212061 <== osd.2 v2:172.21.15.183:6800/3244881082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f960c107140 con 0x7f95fc0aaef0 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] conn(0x7f95fc0aaef0 msgr2=0x7f95fc0ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] conn(0x7f95fc0aaef0 0x7f95fc0ad390 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:36.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f95fc09ff50 msgr2=0x7f95fc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f95fc09ff50 0x7f95fc0a2410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ffca0 tx=0x7f95f8009260 comp rx=0 tx=0).stop 2023-08-25T04:43:36.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 msgr2=0x7f960c19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 --2- 172.21.15.145:0/2297212061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f960c102db0 0x7f960c19e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96000059d0 tx=0x7f9600005720 comp rx=0 tx=0).stop 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f96111d2700 1 -- 172.21.15.145:0/2297212061 reap_dead start 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 shutdown_connections 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.386+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 >> 172.21.15.145:0/2297212061 conn(0x7f960c0fe620 msgr2=0x7f960c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.387+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 shutdown_connections 2023-08-25T04:43:36.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:36.387+0000 7f9612c35700 1 -- 172.21.15.145:0/2297212061 wait complete. 2023-08-25T04:43:36.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:36 smithi145 ceph-mon[97681]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:43:37.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:36 smithi162 ceph-mon[130941]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:43:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:36 smithi183 ceph-mon[131414]: pgmap v290: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:43:37.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 -- 172.21.15.145:0/1855406027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc106940 msgr2=0x7f5ffc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 --2- 172.21.15.145:0/1855406027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc106940 0x7f5ffc106d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009990 tx=0x7f5ff8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 -- 172.21.15.145:0/1855406027 shutdown_connections 2023-08-25T04:43:37.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 --2- 172.21.15.145:0/1855406027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ffc101300 0x7f5ffc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 --2- 172.21.15.145:0/1855406027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc106940 0x7f5ffc106d20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 --2- 172.21.15.145:0/1855406027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ffc100940 0x7f5ffc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 -- 172.21.15.145:0/1855406027 >> 172.21.15.145:0/1855406027 conn(0x7f5ffc0fc410 msgr2=0x7f5ffc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 -- 172.21.15.145:0/1855406027 shutdown_connections 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.340+0000 7f60046ab700 1 -- 172.21.15.145:0/1855406027 wait complete. 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 Processor -- start 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 -- start start 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 0x7f5ffc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:37.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ffc101300 0x7f5ffc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:37.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ffc106940 0x7f5ffc196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:37.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc196ed0 con 0x7f5ffc106940 2023-08-25T04:43:37.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc197040 con 0x7f5ffc101300 2023-08-25T04:43:37.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.341+0000 7f60046ab700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc197200 con 0x7f5ffc100940 2023-08-25T04:43:37.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 0x7f5ffc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6001c46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ffc101300 0x7f5ffc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002c48700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ffc106940 0x7f5ffc196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 0x7f5ffc19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56898/0 (socket says 172.21.15.145:56898) 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 -- 172.21.15.145:0/2696094885 learned_addr learned my addr 172.21.15.145:0/2696094885 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 -- 172.21.15.145:0/2696094885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ffc101300 msgr2=0x7f5ffc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ffc101300 0x7f5ffc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 -- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ffc106940 msgr2=0x7f5ffc196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ffc106940 0x7f5ffc196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80095f0 con 0x7f5ffc100940 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.342+0000 7f6002447700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 0x7f5ffc19c7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff8004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.343+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff801d070 con 0x7f5ffc100940 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.343+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff8021410 con 0x7f5ffc100940 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.343+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc197480 con 0x7f5ffc100940 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.343+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8017650 con 0x7f5ffc100940 2023-08-25T04:43:37.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.343+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc1a37d0 con 0x7f5ffc100940 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.344+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ffc04e620 con 0x7f5ffc100940 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.346+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5ff8021580 con 0x7f5ffc100940 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.347+0000 7f5ff37fe700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5fe809ff50 0x7f5fe80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.347+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5ff80cd1b0 con 0x7f5ffc100940 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f6001c46700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5fe809ff50 0x7f5fe80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:37.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f5ff37fe700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] conn(0x7f5fe80aaef0 0x7f5fe80ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5fe80ada60 con 0x7f5fe80aaef0 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff80cd520 con 0x7f5ffc100940 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f6002c48700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] conn(0x7f5fe80aaef0 0x7f5fe80ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.348+0000 7f6001c46700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5fe809ff50 0x7f5fe80a2410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0fed50 tx=0x7f5fec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.349+0000 7f6002c48700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] conn(0x7f5fe80aaef0 0x7f5fe80ad390 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:37.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.349+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== osd.3 v2:172.21.15.145:6810/3785323151 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5fe80ada60 con 0x7f5fe80aaef0 2023-08-25T04:43:37.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.366+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 --> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ffc110be0 con 0x7f5fe80aaef0 2023-08-25T04:43:37.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.366+0000 7f5ff37fe700 1 -- 172.21.15.145:0/2696094885 <== osd.3 v2:172.21.15.145:6810/3785323151 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5ff400f040 con 0x7f5fe80aaef0 2023-08-25T04:43:37.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] conn(0x7f5fe80aaef0 msgr2=0x7f5fe80ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] conn(0x7f5fe80aaef0 0x7f5fe80ad390 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:37.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5fe809ff50 msgr2=0x7f5fe80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5fe809ff50 0x7f5fe80a2410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0fed50 tx=0x7f5fec009260 comp rx=0 tx=0).stop 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 msgr2=0x7f5ffc19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 --2- 172.21.15.145:0/2696094885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ffc100940 0x7f5ffc19c7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff8004790 comp rx=0 tx=0).stop 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f6002c48700 1 -- 172.21.15.145:0/2696094885 reap_dead start 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 shutdown_connections 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 >> 172.21.15.145:0/2696094885 conn(0x7f5ffc0fc410 msgr2=0x7f5ffc0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 shutdown_connections 2023-08-25T04:43:37.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:37.367+0000 7f60046ab700 1 -- 172.21.15.145:0/2696094885 wait complete. 2023-08-25T04:43:39.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:38 smithi145 ceph-mon[97681]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-08-25T04:43:39.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 -- 172.21.15.145:0/1429915313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 msgr2=0x7f898810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 --2- 172.21.15.145:0/1429915313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 0x7f898810d810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f898000b0a0 tx=0x7f898000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 -- 172.21.15.145:0/1429915313 shutdown_connections 2023-08-25T04:43:39.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 --2- 172.21.15.145:0/1429915313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 0x7f898810d810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 --2- 172.21.15.145:0/1429915313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988106940 0x7f8988106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 --2- 172.21.15.145:0/1429915313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 -- 172.21.15.145:0/1429915313 >> 172.21.15.145:0/1429915313 conn(0x7f89880fc410 msgr2=0x7f89880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:39.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 -- 172.21.15.145:0/1429915313 shutdown_connections 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.819+0000 7f898f478700 1 -- 172.21.15.145:0/1429915313 wait complete. 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.820+0000 7f898f478700 1 Processor -- start 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.820+0000 7f898f478700 1 -- start start 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 0x7f8988073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988106940 0x7f89880736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:39.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881a4fb0 con 0x7f8988101300 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881a5120 con 0x7f8988106940 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898f478700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881a52e0 con 0x7f8988100940 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898d214700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898d214700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56906/0 (socket says 172.21.15.145:56906) 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898d214700 1 -- 172.21.15.145:0/504494522 learned_addr learned my addr 172.21.15.145:0/504494522 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898ca13700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 0x7f8988073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:39.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.821+0000 7f898da15700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988106940 0x7f89880736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 -- 172.21.15.145:0/504494522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988106940 msgr2=0x7f89880736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988106940 0x7f89880736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 -- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 msgr2=0x7f8988073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8988101300 0x7f8988073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980009d00 con 0x7f8988100940 2023-08-25T04:43:39.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898d214700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988072c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9a0 tx=0x7f898400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984009790 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89881a55c0 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8984010430 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984018680 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.822+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89881a5ac0 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.823+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f898804e620 con 0x7f8988100940 2023-08-25T04:43:39.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.825+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f89840105a0 con 0x7f8988100940 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.826+0000 7f897e7fc700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f897409ff50 0x7f89740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8984014070 con 0x7f8988100940 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f898ca13700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f897409ff50 0x7f89740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f897e7fc700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] conn(0x7f89740aaef0 0x7f89740ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89740ada60 con 0x7f89740aaef0 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89840cdad0 con 0x7f8988100940 2023-08-25T04:43:39.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f898da15700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] conn(0x7f89740aaef0 0x7f89740ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:39.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.827+0000 7f898ca13700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f897409ff50 0x7f89740a2410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8988074410 tx=0x7f8978009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:39.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.828+0000 7f898da15700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] conn(0x7f89740aaef0 0x7f89740ad390 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:39.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.828+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== osd.0 v2:172.21.15.145:6802/3430955976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f89740ada60 con 0x7f89740aaef0 2023-08-25T04:43:39.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.845+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 --> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8988110c30 con 0x7f89740aaef0 2023-08-25T04:43:39.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.845+0000 7f897e7fc700 1 -- 172.21.15.145:0/504494522 <== osd.0 v2:172.21.15.145:6802/3430955976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8980009b80 con 0x7f89740aaef0 2023-08-25T04:43:39.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.845+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] conn(0x7f89740aaef0 msgr2=0x7f89740ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.845+0000 7f898f478700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] conn(0x7f89740aaef0 0x7f89740ad390 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f897409ff50 msgr2=0x7f89740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f897409ff50 0x7f89740a2410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8988074410 tx=0x7f8978009260 comp rx=0 tx=0).stop 2023-08-25T04:43:39.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 msgr2=0x7f8988072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:39.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 --2- 172.21.15.145:0/504494522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8988100940 0x7f8988072c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9a0 tx=0x7f898400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:39.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898da15700 1 -- 172.21.15.145:0/504494522 reap_dead start 2023-08-25T04:43:39.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 shutdown_connections 2023-08-25T04:43:39.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 >> 172.21.15.145:0/504494522 conn(0x7f89880fc410 msgr2=0x7f89880fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:39.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 shutdown_connections 2023-08-25T04:43:39.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:38.846+0000 7f898f478700 1 -- 172.21.15.145:0/504494522 wait complete. 2023-08-25T04:43:39.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:38 smithi162 ceph-mon[130941]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-08-25T04:43:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:38 smithi183 ceph-mon[131414]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-08-25T04:43:39.578 INFO:teuthology.orchestra.run.smithi145.stdout:77309411388 2023-08-25T04:43:39.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-08-25T04:43:39.751 INFO:teuthology.orchestra.run.smithi145.stdout:111669149750 2023-08-25T04:43:39.752 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-08-25T04:43:40.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 -- 172.21.15.145:0/996538978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 msgr2=0x7f22bc105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:40.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 --2- 172.21.15.145:0/996538978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 0x7f22bc105810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009920 tx=0x7f22b0009c30 comp rx=0 tx=0).stop 2023-08-25T04:43:40.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 -- 172.21.15.145:0/996538978 shutdown_connections 2023-08-25T04:43:40.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 --2- 172.21.15.145:0/996538978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 0x7f22bc10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 --2- 172.21.15.145:0/996538978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 --2- 172.21.15.145:0/996538978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 0x7f22bc105810 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 -- 172.21.15.145:0/996538978 >> 172.21.15.145:0/996538978 conn(0x7f22bc076920 msgr2=0x7f22bc076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 -- 172.21.15.145:0/996538978 shutdown_connections 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.770+0000 7f22c106e700 1 -- 172.21.15.145:0/996538978 wait complete. 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 Processor -- start 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 -- start start 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 0x7f22bc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:40.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 0x7f22bc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc196f80 con 0x7f22bc0688e0 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1970f0 con 0x7f22bc105d50 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.772+0000 7f22c106e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1972b0 con 0x7f22bc105390 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47682/0 (socket says 172.21.15.145:47682) 2023-08-25T04:43:40.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 -- 172.21.15.145:0/2186067908 learned_addr learned my addr 172.21.15.145:0/2186067908 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:40.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bb59e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 0x7f22bc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:40.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 -- 172.21.15.145:0/2186067908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 msgr2=0x7f22bc19cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T04:43:40.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22bc105390 0x7f22bc19cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 -- 172.21.15.145:0/2186067908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 msgr2=0x7f22bc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:40.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 0x7f22bc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bad9d700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009520 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.773+0000 7f22bb59e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22bc105d50 0x7f22bc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.774+0000 7f22bad9d700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc19c890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e9c0 tx=0x7f22ac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.774+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00cbd0 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.774+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ac00eeb0 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.774+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac005580 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.774+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc197590 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.775+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc06a510 con 0x7f22bc0688e0 2023-08-25T04:43:40.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.776+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22bc04e620 con 0x7f22bc0688e0 2023-08-25T04:43:40.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.778+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f22ac004750 con 0x7f22bc0688e0 2023-08-25T04:43:40.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.780+0000 7f22a3fff700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f22a409ffd0 0x7f22a40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:40.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.781+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f22ac014070 con 0x7f22bc0688e0 2023-08-25T04:43:40.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.781+0000 7f22a3fff700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] conn(0x7f22a40aaf70 0x7f22a40ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.781+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22a40adae0 con 0x7f22a40aaf70 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.781+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ac0cbf60 con 0x7f22bc0688e0 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.781+0000 7f22ba59c700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f22a409ffd0 0x7f22a40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.782+0000 7f22bb59e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] conn(0x7f22a40aaf70 0x7f22a40ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.782+0000 7f22ba59c700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f22a409ffd0 0x7f22a40a2490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009230 tx=0x7f22b00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:40.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.782+0000 7f22bb59e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] conn(0x7f22a40aaf70 0x7f22a40ad410 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.783+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== osd.6 v2:172.21.15.145:6818/141129064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f22a40adae0 con 0x7f22a40aaf70 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.801+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 --> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22bc0fe020 con 0x7f22a40aaf70 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.802+0000 7f22a3fff700 1 -- 172.21.15.145:0/2186067908 <== osd.6 v2:172.21.15.145:6818/141129064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f22bc0fe020 con 0x7f22a40aaf70 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.802+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] conn(0x7f22a40aaf70 msgr2=0x7f22a40ad410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.802+0000 7f22c106e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] conn(0x7f22a40aaf70 0x7f22a40ad410 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f22a409ffd0 msgr2=0x7f22a40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:40.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f22a409ffd0 0x7f22a40a2490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009230 tx=0x7f22b00058e0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 msgr2=0x7f22bc19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 --2- 172.21.15.145:0/2186067908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22bc0688e0 0x7f22bc19c890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e9c0 tx=0x7f22ac00ecd0 comp rx=0 tx=0).stop 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22bb59e700 1 -- 172.21.15.145:0/2186067908 reap_dead start 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 shutdown_connections 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 >> 172.21.15.145:0/2186067908 conn(0x7f22bc076920 msgr2=0x7f22bc0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 shutdown_connections 2023-08-25T04:43:40.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:39.803+0000 7f22c106e700 1 -- 172.21.15.145:0/2186067908 wait complete. 2023-08-25T04:43:40.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:40 smithi145 ceph-mon[97681]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:40.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:40 smithi162 ceph-mon[130941]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:40.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:40 smithi183 ceph-mon[131414]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:41.048 INFO:teuthology.orchestra.run.smithi145.stdout:34359738438 2023-08-25T04:43:41.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-08-25T04:43:41.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 -- 172.21.15.145:0/2408989134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c0ffc10 msgr2=0x7fcf5c0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 --2- 172.21.15.145:0/2408989134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0ffff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009990 tx=0x7fcf4c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 -- 172.21.15.145:0/2408989134 shutdown_connections 2023-08-25T04:43:41.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 --2- 172.21.15.145:0/2408989134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5c101d10 0x7fcf5c10c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 --2- 172.21.15.145:0/2408989134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0ffff0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 --2- 172.21.15.145:0/2408989134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c101350 0x7fcf5c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 -- 172.21.15.145:0/2408989134 >> 172.21.15.145:0/2408989134 conn(0x7fcf5c076910 msgr2=0x7fcf5c076d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 -- 172.21.15.145:0/2408989134 shutdown_connections 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.534+0000 7fcf629dc700 1 -- 172.21.15.145:0/2408989134 wait complete. 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.536+0000 7fcf629dc700 1 Processor -- start 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.536+0000 7fcf629dc700 1 -- start start 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c101350 0x7fcf5c0fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5c101d10 0x7fcf5c0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a4f00 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a5070 con 0x7fcf5c101350 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf629dc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a5230 con 0x7fcf5c101d10 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47698/0 (socket says 172.21.15.145:47698) 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf5bfff700 1 -- 172.21.15.145:0/1488507273 learned_addr learned my addr 172.21.15.145:0/1488507273 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf5b7fe700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c101350 0x7fcf5c0fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.537+0000 7fcf60f79700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5c101d10 0x7fcf5c0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 -- 172.21.15.145:0/1488507273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5c101d10 msgr2=0x7fcf5c0fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf5c101d10 0x7fcf5c0fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 -- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c101350 msgr2=0x7fcf5c0fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c101350 0x7fcf5c0fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0095f0 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5b7fe700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5c101350 0x7fcf5c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.538+0000 7fcf5bfff700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0fc8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0095c0 tx=0x7fcf4c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.540+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c01d070 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.540+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf4c005050 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.540+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004b30 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.540+0000 7fcf629dc700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c1a54b0 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.540+0000 7fcf629dc700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c1a5900 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.541+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf3c000fc0 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.544+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcf4c005430 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf597fa700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf4409ffd0 0x7fcf440a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcf4c0ccaf0 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf5b7fe700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf4409ffd0 0x7fcf440a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf597fa700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] conn(0x7fcf440aaf70 0x7fcf440ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf440adae0 con 0x7fcf440aaf70 2023-08-25T04:43:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.545+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf4c0cce60 con 0x7fcf5c0ffc10 2023-08-25T04:43:41.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.546+0000 7fcf60f79700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] conn(0x7fcf440aaf70 0x7fcf440ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.546+0000 7fcf5b7fe700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf4409ffd0 0x7fcf440a2490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf50005950 tx=0x7fcf500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.547+0000 7fcf60f79700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] conn(0x7fcf440aaf70 0x7fcf440ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.547+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== osd.4 v2:172.21.15.162:6808/739956893 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcf440adae0 con 0x7fcf440aaf70 2023-08-25T04:43:41.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.568+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 --> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf3c002d40 con 0x7fcf440aaf70 2023-08-25T04:43:41.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.569+0000 7fcf597fa700 1 -- 172.21.15.145:0/1488507273 <== osd.4 v2:172.21.15.162:6808/739956893 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcf3c002d40 con 0x7fcf440aaf70 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] conn(0x7fcf440aaf70 msgr2=0x7fcf440ad410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] conn(0x7fcf440aaf70 0x7fcf440ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf4409ffd0 msgr2=0x7fcf440a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf4409ffd0 0x7fcf440a2490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf50005950 tx=0x7fcf500058e0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 msgr2=0x7fcf5c0fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 --2- 172.21.15.145:0/1488507273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf5c0ffc10 0x7fcf5c0fc8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0095c0 tx=0x7fcf4c00f690 comp rx=0 tx=0).stop 2023-08-25T04:43:41.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf60f79700 1 -- 172.21.15.145:0/1488507273 reap_dead start 2023-08-25T04:43:41.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 shutdown_connections 2023-08-25T04:43:41.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 >> 172.21.15.145:0/1488507273 conn(0x7fcf5c076910 msgr2=0x7fcf5c068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 shutdown_connections 2023-08-25T04:43:41.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.570+0000 7fcf42ffd700 1 -- 172.21.15.145:0/1488507273 wait complete. 2023-08-25T04:43:41.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 -- 172.21.15.145:0/2382626056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c101300 msgr2=0x7f130c10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 --2- 172.21.15.145:0/2382626056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c101300 0x7f130c10d810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1304009f90 tx=0x7f130400b330 comp rx=0 tx=0).stop 2023-08-25T04:43:41.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 -- 172.21.15.145:0/2382626056 shutdown_connections 2023-08-25T04:43:41.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 --2- 172.21.15.145:0/2382626056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c101300 0x7f130c10d810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 --2- 172.21.15.145:0/2382626056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 0x7f130c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 --2- 172.21.15.145:0/2382626056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c100940 0x7f130c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 -- 172.21.15.145:0/2382626056 >> 172.21.15.145:0/2382626056 conn(0x7f130c0fc410 msgr2=0x7f130c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 -- 172.21.15.145:0/2382626056 shutdown_connections 2023-08-25T04:43:41.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.552+0000 7f1311783700 1 -- 172.21.15.145:0/2382626056 wait complete. 2023-08-25T04:43:41.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.553+0000 7f1311783700 1 Processor -- start 2023-08-25T04:43:41.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.553+0000 7f1311783700 1 -- start start 2023-08-25T04:43:41.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c100940 0x7f130c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c101300 0x7f130c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 0x7f130c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19dcf0 con 0x7f130c100940 2023-08-25T04:43:41.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19de60 con 0x7f130c106940 2023-08-25T04:43:41.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f1311783700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19e020 con 0x7f130c101300 2023-08-25T04:43:41.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c100940 0x7f130c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 0x7f130c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c101300 0x7f130c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c100940 0x7f130c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47746/0 (socket says 172.21.15.145:47746) 2023-08-25T04:43:41.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130affd700 1 -- 172.21.15.145:0/339534477 learned_addr learned my addr 172.21.15.145:0/339534477 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130a7fc700 1 -- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 msgr2=0x7f130c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130a7fc700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 0x7f130c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.554+0000 7f130a7fc700 1 -- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c100940 msgr2=0x7f130c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f130a7fc700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f130c100940 0x7f130c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f130a7fc700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0095f0 con 0x7f130c101300 2023-08-25T04:43:41.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f130b7fe700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f130c106940 0x7f130c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:41.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f130a7fc700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c101300 0x7f130c199270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f130000e9f0 tx=0x7f130000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000c9c0 con 0x7f130c101300 2023-08-25T04:43:41.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1304009c40 con 0x7f130c101300 2023-08-25T04:43:41.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300010430 con 0x7f130c101300 2023-08-25T04:43:41.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300018680 con 0x7f130c101300 2023-08-25T04:43:41.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.555+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c1a5990 con 0x7f130c101300 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.556+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f130c04e620 con 0x7f130c101300 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.559+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f1300010860 con 0x7f130c101300 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.560+0000 7f12f3fff700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f12f409ff50 0x7f12f40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1300014070 con 0x7f130c101300 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f130affd700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f12f409ff50 0x7f12f40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f12f3fff700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] conn(0x7f12f40aaef0 0x7f12f40ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12f40ada60 con 0x7f12f40aaef0 2023-08-25T04:43:41.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13000cbd20 con 0x7f130c101300 2023-08-25T04:43:41.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f130b7fe700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] conn(0x7f12f40aaef0 0x7f12f40ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.561+0000 7f130affd700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f12f409ff50 0x7f12f40a2410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005d10 tx=0x7f12fc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.562+0000 7f130b7fe700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] conn(0x7f12f40aaef0 0x7f12f40ad390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.564+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== osd.1 v2:172.21.15.162:6800/4196277224 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f12f40ada60 con 0x7f12f40aaef0 2023-08-25T04:43:41.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.593+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 --> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f130c110c20 con 0x7f12f40aaef0 2023-08-25T04:43:41.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f12f3fff700 1 -- 172.21.15.145:0/339534477 <== osd.1 v2:172.21.15.162:6800/4196277224 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1304007bd0 con 0x7f12f40aaef0 2023-08-25T04:43:41.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] conn(0x7f12f40aaef0 msgr2=0x7f12f40ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] conn(0x7f12f40aaef0 0x7f12f40ad390 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f12f409ff50 msgr2=0x7f12f40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f12f409ff50 0x7f12f40a2410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005d10 tx=0x7f12fc009310 comp rx=0 tx=0).stop 2023-08-25T04:43:41.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c101300 msgr2=0x7f130c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f1311783700 1 --2- 172.21.15.145:0/339534477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130c101300 0x7f130c199270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f130000e9f0 tx=0x7f130000ed00 comp rx=0 tx=0).stop 2023-08-25T04:43:41.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.594+0000 7f130b7fe700 1 -- 172.21.15.145:0/339534477 reap_dead start 2023-08-25T04:43:41.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.595+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 shutdown_connections 2023-08-25T04:43:41.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.595+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 >> 172.21.15.145:0/339534477 conn(0x7f130c0fc410 msgr2=0x7f130c10bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.595+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 shutdown_connections 2023-08-25T04:43:41.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.595+0000 7f1311783700 1 -- 172.21.15.145:0/339534477 wait complete. 2023-08-25T04:43:41.656 INFO:teuthology.orchestra.run.smithi145.stdout:197568495660 2023-08-25T04:43:41.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-08-25T04:43:41.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 -- 172.21.15.145:0/279324413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c111950 msgr2=0x7fc89c113d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 --2- 172.21.15.145:0/279324413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c111950 0x7fc89c113d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc89400b0a0 tx=0x7fc89400b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 -- 172.21.15.145:0/279324413 shutdown_connections 2023-08-25T04:43:41.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 --2- 172.21.15.145:0/279324413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c111950 0x7fc89c113d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 --2- 172.21.15.145:0/279324413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 --2- 172.21.15.145:0/279324413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c0735e0 0x7fc89c111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 -- 172.21.15.145:0/279324413 >> 172.21.15.145:0/279324413 conn(0x7fc89c0fc410 msgr2=0x7fc89c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 -- 172.21.15.145:0/279324413 shutdown_connections 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.415+0000 7fc8a4156700 1 -- 172.21.15.145:0/279324413 wait complete. 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 Processor -- start 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 -- start start 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c0735e0 0x7fc89c10ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c111950 0x7fc89c109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c10a2b0 con 0x7fc89c0735e0 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c10a420 con 0x7fc89c072cc0 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.417+0000 7fc8a4156700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c10a5e0 con 0x7fc89c111950 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c10ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:38468/0 (socket says 172.21.15.145:38468) 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 -- 172.21.15.145:0/3372822544 learned_addr learned my addr 172.21.15.145:0/3372822544 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a16f1700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c0735e0 0x7fc89c10ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a26f3700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c111950 0x7fc89c109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 -- 172.21.15.145:0/3372822544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c111950 msgr2=0x7fc89c109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c111950 0x7fc89c109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 -- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c0735e0 msgr2=0x7fc89c10ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c0735e0 0x7fc89c10ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a1ef2700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c009520 con 0x7fc89c072cc0 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a26f3700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc89c111950 0x7fc89c109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.418+0000 7fc8a16f1700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc89c0735e0 0x7fc89c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:41.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.419+0000 7fc8a1ef2700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c10ea10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9a0 tx=0x7fc89800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.419+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898009790 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.419+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc894009d00 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.419+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89800cbd0 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.419+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898018680 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.420+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c10d950 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.420+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc89c117190 con 0x7fc89c072cc0 2023-08-25T04:43:41.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.423+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc89800cd40 con 0x7fc89c072cc0 2023-08-25T04:43:41.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc892ffd700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc88809ffd0 0x7fc8880a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc898014070 con 0x7fc89c072cc0 2023-08-25T04:43:41.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc8a16f1700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc88809ffd0 0x7fc8880a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc892ffd700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] conn(0x7fc8880aaf70 0x7fc8880ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8880adae0 con 0x7fc8880aaf70 2023-08-25T04:43:41.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.425+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8980cc270 con 0x7fc89c072cc0 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.426+0000 7fc8a26f3700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] conn(0x7fc8880aaf70 0x7fc8880ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.426+0000 7fc8a16f1700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc88809ffd0 0x7fc8880a2490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc89c10b390 tx=0x7fc88c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.426+0000 7fc8a26f3700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] conn(0x7fc8880aaf70 0x7fc8880ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.427+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== osd.10 v2:172.21.15.162:6824/3093586037 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc8880adae0 con 0x7fc8880aaf70 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.444+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 --> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc89c04e620 con 0x7fc8880aaf70 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.444+0000 7fc892ffd700 1 -- 172.21.15.145:0/3372822544 <== osd.10 v2:172.21.15.162:6824/3093586037 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc89c04e620 con 0x7fc8880aaf70 2023-08-25T04:43:41.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] conn(0x7fc8880aaf70 msgr2=0x7fc8880ad410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] conn(0x7fc8880aaf70 0x7fc8880ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc88809ffd0 msgr2=0x7fc8880a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc88809ffd0 0x7fc8880a2490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc89c10b390 tx=0x7fc88c009260 comp rx=0 tx=0).stop 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 msgr2=0x7fc89c10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 --2- 172.21.15.145:0/3372822544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc89c072cc0 0x7fc89c10ea10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9a0 tx=0x7fc89800ed60 comp rx=0 tx=0).stop 2023-08-25T04:43:41.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a26f3700 1 -- 172.21.15.145:0/3372822544 reap_dead start 2023-08-25T04:43:41.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 shutdown_connections 2023-08-25T04:43:41.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 >> 172.21.15.145:0/3372822544 conn(0x7fc89c0fc410 msgr2=0x7fc89c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 shutdown_connections 2023-08-25T04:43:41.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.445+0000 7fc8a4156700 1 -- 172.21.15.145:0/3372822544 wait complete. 2023-08-25T04:43:41.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 -- 172.21.15.145:0/856244449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990069760 msgr2=0x7ff99010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 --2- 172.21.15.145:0/856244449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990069760 0x7ff99010d840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b0a0 tx=0x7ff98800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 -- 172.21.15.145:0/856244449 shutdown_connections 2023-08-25T04:43:41.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 --2- 172.21.15.145:0/856244449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990069760 0x7ff99010d840 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 --2- 172.21.15.145:0/856244449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990106a20 0x7ff990106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 --2- 172.21.15.145:0/856244449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 0x7ff990069220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 -- 172.21.15.145:0/856244449 >> 172.21.15.145:0/856244449 conn(0x7ff9900769a0 msgr2=0x7ff990076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 -- 172.21.15.145:0/856244449 shutdown_connections 2023-08-25T04:43:41.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.965+0000 7ff997ba2700 1 -- 172.21.15.145:0/856244449 wait complete. 2023-08-25T04:43:41.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.967+0000 7ff997ba2700 1 Processor -- start 2023-08-25T04:43:41.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.967+0000 7ff997ba2700 1 -- start start 2023-08-25T04:43:41.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.967+0000 7ff997ba2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 0x7ff9900ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.967+0000 7ff997ba2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 0x7ff9900ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.967+0000 7ff997ba2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990106a20 0x7ff990100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff997ba2700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a4f40 con 0x7ff990106a20 2023-08-25T04:43:41.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff997ba2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a50b0 con 0x7ff990069760 2023-08-25T04:43:41.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff997ba2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901a5270 con 0x7ff990068da0 2023-08-25T04:43:41.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99593e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 0x7ff9900ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99613f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990106a20 0x7ff990100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 0x7ff9900ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 0x7ff9900ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:38438/0 (socket says 172.21.15.145:38438) 2023-08-25T04:43:41.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 -- 172.21.15.145:0/677665623 learned_addr learned my addr 172.21.15.145:0/677665623 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:41.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 -- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 msgr2=0x7ff9900ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 0x7ff9900ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 -- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990106a20 msgr2=0x7ff990100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff990106a20 0x7ff990100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99513d700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff988009d00 con 0x7ff990069760 2023-08-25T04:43:41.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.968+0000 7ff99593e700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990068da0 0x7ff9900ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:41.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff99513d700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 0x7ff9900ffde0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e9a0 tx=0x7ff98000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980009790 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901a5550 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff980010430 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980018680 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.969+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901a5aa0 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.970+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff99004e620 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.972+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ff9800105a0 con 0x7ff990069760 2023-08-25T04:43:41.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.973+0000 7ff986ffd700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff97c09ffd0 0x7ff97c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.973+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff980014070 con 0x7ff990069760 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff986ffd700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] conn(0x7ff97c0aaf70 0x7ff97c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff99593e700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff97c09ffd0 0x7ff97c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff97c0adae0 con 0x7ff97c0aaf70 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9800cbd00 con 0x7ff990069760 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff99613f700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] conn(0x7ff97c0aaf70 0x7ff97c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.974+0000 7ff99593e700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff97c09ffd0 0x7ff97c0a2490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff98c011490 tx=0x7ff98c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.975+0000 7ff99613f700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] conn(0x7ff97c0aaf70 0x7ff97c0ad410 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.976+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== osd.5 v2:172.21.15.183:6808/15251435 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff97c0adae0 con 0x7ff97c0aaf70 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.998+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 --> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff990110bd0 con 0x7ff97c0aaf70 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:40.999+0000 7ff986ffd700 1 -- 172.21.15.145:0/677665623 <== osd.5 v2:172.21.15.183:6808/15251435 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff988009b80 con 0x7ff97c0aaf70 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] conn(0x7ff97c0aaf70 msgr2=0x7ff97c0ad410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] conn(0x7ff97c0aaf70 0x7ff97c0ad410 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff97c09ffd0 msgr2=0x7ff97c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff97c09ffd0 0x7ff97c0a2490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff98c011490 tx=0x7ff98c009260 comp rx=0 tx=0).stop 2023-08-25T04:43:41.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 msgr2=0x7ff9900ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 --2- 172.21.15.145:0/677665623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff990069760 0x7ff9900ffde0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e9a0 tx=0x7ff98000ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff99613f700 1 -- 172.21.15.145:0/677665623 reap_dead start 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 shutdown_connections 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 >> 172.21.15.145:0/677665623 conn(0x7ff9900769a0 msgr2=0x7ff990107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 shutdown_connections 2023-08-25T04:43:41.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.000+0000 7ff997ba2700 1 -- 172.21.15.145:0/677665623 wait complete. 2023-08-25T04:43:41.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 -- 172.21.15.145:0/4205663805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774106940 msgr2=0x7f0774106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 --2- 172.21.15.145:0/4205663805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774106940 0x7f0774106d20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0764009990 tx=0x7f0764009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 -- 172.21.15.145:0/4205663805 shutdown_connections 2023-08-25T04:43:41.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 --2- 172.21.15.145:0/4205663805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0774101300 0x7f077410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 --2- 172.21.15.145:0/4205663805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774106940 0x7f0774106d20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 --2- 172.21.15.145:0/4205663805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774100940 0x7f0774100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 -- 172.21.15.145:0/4205663805 >> 172.21.15.145:0/4205663805 conn(0x7f07740fc410 msgr2=0x7f07740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 -- 172.21.15.145:0/4205663805 shutdown_connections 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.184+0000 7f077a58e700 1 -- 172.21.15.145:0/4205663805 wait complete. 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.185+0000 7f077a58e700 1 Processor -- start 2023-08-25T04:43:41.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.185+0000 7f077a58e700 1 -- start start 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774100940 0x7f077419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 0x7f077419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0774106940 0x7f0774196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774196e90 con 0x7f0774101300 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774197000 con 0x7f0774106940 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f077a58e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07741971c0 con 0x7f0774100940 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 0x7f077419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 0x7f077419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47724/0 (socket says 172.21.15.145:47724) 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 -- 172.21.15.145:0/186690551 learned_addr learned my addr 172.21.15.145:0/186690551 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f0773fff700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774100940 0x7f077419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f0778b2b700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0774106940 0x7f0774196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 -- 172.21.15.145:0/186690551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774100940 msgr2=0x7f077419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774100940 0x7f077419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 -- 172.21.15.145:0/186690551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0774106940 msgr2=0x7f0774196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0774106940 0x7f0774196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.186+0000 7f07737fe700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009520 con 0x7f0774101300 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f07737fe700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 0x7f077419cce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0768009f60 tx=0x7f0768009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768013be0 con 0x7f0774101300 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640095f0 con 0x7f0774101300 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07680044e0 con 0x7f0774101300 2023-08-25T04:43:41.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768003680 con 0x7f0774101300 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.187+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0774072d20 con 0x7f0774101300 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.188+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0774197690 con 0x7f0774101300 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.190+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f076801b440 con 0x7f0774101300 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f07717fa700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f075c09ff50 0x7f075c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f07680d0070 con 0x7f0774101300 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f0773fff700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f075c09ff50 0x7f075c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f07717fa700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] conn(0x7f075c0aaef0 0x7f075c0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f075c0ada60 con 0x7f075c0aaef0 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.192+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07680d03e0 con 0x7f0774101300 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.193+0000 7f0778b2b700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] conn(0x7f075c0aaef0 0x7f075c0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.193+0000 7f0773fff700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f075c09ff50 0x7f075c0a2410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f076400b2e0 tx=0x7f0764005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.193+0000 7f0778b2b700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] conn(0x7f075c0aaef0 0x7f075c0ad390 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:41.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.194+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== osd.9 v2:172.21.15.145:6826/4228444400 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f075c0ada60 con 0x7f075c0aaef0 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.213+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 --> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0774110be0 con 0x7f075c0aaef0 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f07717fa700 1 -- 172.21.15.145:0/186690551 <== osd.9 v2:172.21.15.145:6826/4228444400 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f076c009cd0 con 0x7f075c0aaef0 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] conn(0x7f075c0aaef0 msgr2=0x7f075c0ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] conn(0x7f075c0aaef0 0x7f075c0ad390 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f075c09ff50 msgr2=0x7f075c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f075c09ff50 0x7f075c0a2410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f076400b2e0 tx=0x7f0764005960 comp rx=0 tx=0).stop 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 msgr2=0x7f077419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:41.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 --2- 172.21.15.145:0/186690551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0774101300 0x7f077419cce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0768009f60 tx=0x7f0768009f90 comp rx=0 tx=0).stop 2023-08-25T04:43:41.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f0778b2b700 1 -- 172.21.15.145:0/186690551 reap_dead start 2023-08-25T04:43:41.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 shutdown_connections 2023-08-25T04:43:41.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 >> 172.21.15.145:0/186690551 conn(0x7f07740fc410 msgr2=0x7f0774104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:41.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 shutdown_connections 2023-08-25T04:43:41.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:41.214+0000 7f077a58e700 1 -- 172.21.15.145:0/186690551 wait complete. 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 -- 172.21.15.145:0/4188421143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481068f0 msgr2=0x7fae48106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/4188421143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481068f0 0x7fae48106cd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae44009990 tx=0x7fae44009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 -- 172.21.15.145:0/4188421143 shutdown_connections 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/4188421143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae481012b0 0x7fae4810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/4188421143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481068f0 0x7fae48106cd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/4188421143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae481008f0 0x7fae48100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 -- 172.21.15.145:0/4188421143 >> 172.21.15.145:0/4188421143 conn(0x7fae480fc420 msgr2=0x7fae480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:42.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 -- 172.21.15.145:0/4188421143 shutdown_connections 2023-08-25T04:43:42.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.014+0000 7fae4f5c0700 1 -- 172.21.15.145:0/4188421143 wait complete. 2023-08-25T04:43:42.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.015+0000 7fae4f5c0700 1 Processor -- start 2023-08-25T04:43:42.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 -- start start 2023-08-25T04:43:42.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 0x7fae4819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae481012b0 0x7fae4819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae481068f0 0x7fae48198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481990f0 con 0x7fae481008f0 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48199260 con 0x7fae481068f0 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.016+0000 7fae4f5c0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48199420 con 0x7fae481012b0 2023-08-25T04:43:42.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 0x7fae4819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4cb5b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae481012b0 0x7fae4819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 0x7fae4819ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47762/0 (socket says 172.21.15.145:47762) 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 -- 172.21.15.145:0/2625477112 learned_addr learned my addr 172.21.15.145:0/2625477112 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4db5d700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae481068f0 0x7fae48198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 -- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae481012b0 msgr2=0x7fae4819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae481012b0 0x7fae4819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 -- 172.21.15.145:0/2625477112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae481068f0 msgr2=0x7fae48198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae481068f0 0x7fae48198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440095f0 con 0x7fae481008f0 2023-08-25T04:43:42.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.017+0000 7fae4d35c700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 0x7fae4819ea00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae440095c0 tx=0x7fae4400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:42.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.018+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4401d070 con 0x7fae481008f0 2023-08-25T04:43:42.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.018+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae44005050 con 0x7fae481008f0 2023-08-25T04:43:42.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.018+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481996a0 con 0x7fae481008f0 2023-08-25T04:43:42.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.018+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44004a90 con 0x7fae481008f0 2023-08-25T04:43:42.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.018+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481a59f0 con 0x7fae481008f0 2023-08-25T04:43:42.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.019+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae4804e620 con 0x7fae481008f0 2023-08-25T04:43:42.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.021+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fae44005430 con 0x7fae481008f0 2023-08-25T04:43:42.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.022+0000 7fae3e7fc700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fae3409ffd0 0x7fae340a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:42.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.022+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fae440cca00 con 0x7fae481008f0 2023-08-25T04:43:42.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.022+0000 7fae4cb5b700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fae3409ffd0 0x7fae340a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:42.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.023+0000 7fae3e7fc700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] conn(0x7fae340aaf70 0x7fae340ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:42.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.023+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae340adae0 con 0x7fae340aaf70 2023-08-25T04:43:42.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.023+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae440cce10 con 0x7fae481008f0 2023-08-25T04:43:42.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.023+0000 7fae4cb5b700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fae3409ffd0 0x7fae340a2490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae48105fd0 tx=0x7fae380073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:42.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.023+0000 7fae4db5d700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] conn(0x7fae340aaf70 0x7fae340ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:42.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.024+0000 7fae4db5d700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] conn(0x7fae340aaf70 0x7fae340ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:42.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.024+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== osd.8 v2:172.21.15.183:6816/3911543971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fae340adae0 con 0x7fae340aaf70 2023-08-25T04:43:42.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.041+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 --> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae480fd3a0 con 0x7fae340aaf70 2023-08-25T04:43:42.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae3e7fc700 1 -- 172.21.15.145:0/2625477112 <== osd.8 v2:172.21.15.183:6816/3911543971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fae480fd3a0 con 0x7fae340aaf70 2023-08-25T04:43:42.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] conn(0x7fae340aaf70 msgr2=0x7fae340ad410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] conn(0x7fae340aaf70 0x7fae340ad410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fae3409ffd0 msgr2=0x7fae340a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fae3409ffd0 0x7fae340a2490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae48105fd0 tx=0x7fae380073d0 comp rx=0 tx=0).stop 2023-08-25T04:43:42.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 msgr2=0x7fae4819ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:42.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 --2- 172.21.15.145:0/2625477112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae481008f0 0x7fae4819ea00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae440095c0 tx=0x7fae4400f690 comp rx=0 tx=0).stop 2023-08-25T04:43:42.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4db5d700 1 -- 172.21.15.145:0/2625477112 reap_dead start 2023-08-25T04:43:42.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 shutdown_connections 2023-08-25T04:43:42.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 >> 172.21.15.145:0/2625477112 conn(0x7fae480fc420 msgr2=0x7fae480fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:42.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 shutdown_connections 2023-08-25T04:43:42.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:42.042+0000 7fae4f5c0700 1 -- 172.21.15.145:0/2625477112 wait complete. 2023-08-25T04:43:43.017 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:42 smithi145 ceph-mon[97681]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:43.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 -- 172.21.15.145:0/4230286204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4300735e0 msgr2=0x7fa430111410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 --2- 172.21.15.145:0/4230286204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4300735e0 0x7fa430111410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098f0 tx=0x7fa418009c00 comp rx=0 tx=0).stop 2023-08-25T04:43:43.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 -- 172.21.15.145:0/4230286204 shutdown_connections 2023-08-25T04:43:43.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 --2- 172.21.15.145:0/4230286204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa430111950 0x7fa430113d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 --2- 172.21.15.145:0/4230286204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 0x7fa4300730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 --2- 172.21.15.145:0/4230286204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4300735e0 0x7fa430111410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 -- 172.21.15.145:0/4230286204 >> 172.21.15.145:0/4230286204 conn(0x7fa4300fc410 msgr2=0x7fa4300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:43.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 -- 172.21.15.145:0/4230286204 shutdown_connections 2023-08-25T04:43:43.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.021+0000 7fa4362a8700 1 -- 172.21.15.145:0/4230286204 wait complete. 2023-08-25T04:43:43.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.022+0000 7fa4362a8700 1 Processor -- start 2023-08-25T04:43:43.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 -- start start 2023-08-25T04:43:43.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 0x7fa43019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:43.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 0x7fa43019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:43.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa430111950 0x7fa4301a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:43.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301a20f0 con 0x7fa4300735e0 2023-08-25T04:43:43.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301a2260 con 0x7fa430072cc0 2023-08-25T04:43:43.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa4362a8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301a2420 con 0x7fa430111950 2023-08-25T04:43:43.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.023+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 0x7fa43019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:43.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 0x7fa43019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:43.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 0x7fa43019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36878/0 (socket says 172.21.15.145:36878) 2023-08-25T04:43:43.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa42ffff700 1 -- 172.21.15.145:0/3997297731 learned_addr learned my addr 172.21.15.145:0/3997297731 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:43.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa434845700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa430111950 0x7fa4301a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 -- 172.21.15.145:0/3997297731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa430111950 msgr2=0x7fa4301a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa430111950 0x7fa4301a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 -- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 msgr2=0x7fa43019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 0x7fa43019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa427fff700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009520 con 0x7fa430072cc0 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa434845700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa430111950 0x7fa4301a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:43:43.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.024+0000 7fa42ffff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4300735e0 0x7fa43019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:43.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa427fff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 0x7fa43019d130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9c0 tx=0x7fa42000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:43.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000c9c0 con 0x7fa430072cc0 2023-08-25T04:43:43.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4301a2700 con 0x7fa430072cc0 2023-08-25T04:43:43.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa420010430 con 0x7fa430072cc0 2023-08-25T04:43:43.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420018680 con 0x7fa430072cc0 2023-08-25T04:43:43.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.025+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4301a9f60 con 0x7fa430072cc0 2023-08-25T04:43:43.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.026+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa430117150 con 0x7fa430072cc0 2023-08-25T04:43:43.030 INFO:teuthology.orchestra.run.smithi145.stdout:304942678042 2023-08-25T04:43:43.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-08-25T04:43:43.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.029+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa42000cb20 con 0x7fa430072cc0 2023-08-25T04:43:43.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42dffb700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa41c09ff50 0x7fa41c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:43.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa420014070 con 0x7fa430072cc0 2023-08-25T04:43:43.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42ffff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa41c09ff50 0x7fa41c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:43.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42dffb700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] conn(0x7fa41c0aaef0 0x7fa41c0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:43.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa41c0ada60 con 0x7fa41c0aaef0 2023-08-25T04:43:43.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.031+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4200cda70 con 0x7fa430072cc0 2023-08-25T04:43:43.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.032+0000 7fa434845700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] conn(0x7fa41c0aaef0 0x7fa41c0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:43.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.032+0000 7fa42ffff700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa41c09ff50 0x7fa41c0a2410 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa43019e750 tx=0x7fa418019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:43.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.032+0000 7fa434845700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] conn(0x7fa41c0aaef0 0x7fa41c0ad390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:43.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.033+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== osd.7 v2:172.21.15.162:6816/3155030948 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa41c0ada60 con 0x7fa41c0aaef0 2023-08-25T04:43:43.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.055+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 --> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa43004e620 con 0x7fa41c0aaef0 2023-08-25T04:43:43.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.055+0000 7fa42dffb700 1 -- 172.21.15.145:0/3997297731 <== osd.7 v2:172.21.15.162:6816/3155030948 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa43004e620 con 0x7fa41c0aaef0 2023-08-25T04:43:43.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] conn(0x7fa41c0aaef0 msgr2=0x7fa41c0ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] conn(0x7fa41c0aaef0 0x7fa41c0ad390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa41c09ff50 msgr2=0x7fa41c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa41c09ff50 0x7fa41c0a2410 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa43019e750 tx=0x7fa418019040 comp rx=0 tx=0).stop 2023-08-25T04:43:43.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 msgr2=0x7fa43019d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:43.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 --2- 172.21.15.145:0/3997297731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa430072cc0 0x7fa43019d130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9c0 tx=0x7fa42000ecd0 comp rx=0 tx=0).stop 2023-08-25T04:43:43.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa434845700 1 -- 172.21.15.145:0/3997297731 reap_dead start 2023-08-25T04:43:43.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 shutdown_connections 2023-08-25T04:43:43.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 >> 172.21.15.145:0/3997297731 conn(0x7fa4300fc410 msgr2=0x7fa4300fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:43.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 shutdown_connections 2023-08-25T04:43:43.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.056+0000 7fa4362a8700 1 -- 172.21.15.145:0/3997297731 wait complete. 2023-08-25T04:43:43.133 INFO:teuthology.orchestra.run.smithi145.stdout:249108103203 2023-08-25T04:43:43.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-08-25T04:43:43.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:42 smithi162 ceph-mon[130941]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:43.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:42 smithi183 ceph-mon[131414]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:43.368 INFO:teuthology.orchestra.run.smithi145.stdout:283467841566 2023-08-25T04:43:43.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-08-25T04:43:43.736 INFO:teuthology.orchestra.run.smithi145.stdout:141733920820 2023-08-25T04:43:43.737 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-08-25T04:43:44.158 INFO:teuthology.orchestra.run.smithi145.stdout:55834574912 2023-08-25T04:43:44.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-08-25T04:43:44.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3841927821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c8072d50 msgr2=0x7fb8c8073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3841927821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c8072d50 0x7fb8c8073130 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009990 tx=0x7fb8b8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3841927821 shutdown_connections 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3841927821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c810d180 0x7fb8c810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3841927821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c8072d50 0x7fb8c8073130 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3841927821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.879+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3841927821 >> 172.21.15.145:0/3841927821 conn(0x7fb8c80fc410 msgr2=0x7fb8c80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.880+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3841927821 shutdown_connections 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.880+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3841927821 wait complete. 2023-08-25T04:43:44.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.881+0000 7fb8ccb55700 1 Processor -- start 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.881+0000 7fb8ccb55700 1 -- start start 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c8072d50 0x7fb8c819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c810d180 0x7fb8c8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8199130 con 0x7fb8c810d180 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81992a0 con 0x7fb8c8073670 2023-08-25T04:43:44.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8ccb55700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8199460 con 0x7fb8c8072d50 2023-08-25T04:43:44.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8c6d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c810d180 0x7fb8c8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:44.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8c659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:44.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8c659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:55132/0 (socket says 172.21.15.145:55132) 2023-08-25T04:43:44.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8c659c700 1 -- 172.21.15.145:0/3656492741 learned_addr learned my addr 172.21.15.145:0/3656492741 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:44.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.882+0000 7fb8be59c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c8072d50 0x7fb8c819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:44.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 -- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c8072d50 msgr2=0x7fb8c819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c8072d50 0x7fb8c819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 -- 172.21.15.145:0/3656492741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 msgr2=0x7fb8c819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80095f0 con 0x7fb8c810d180 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c659c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8c8073670 0x7fb8c819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8be59c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8c8072d50 0x7fb8c819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8c6d9d700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c810d180 0x7fb8c8198ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000efc0 tx=0x7fb8c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0018070 con 0x7fb8c810d180 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c81996e0 con 0x7fb8c810d180 2023-08-25T04:43:44.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8c000f040 con 0x7fb8c810d180 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000cd40 con 0x7fb8c810d180 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.883+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c81a5a30 con 0x7fb8c810d180 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.884+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8c8112da0 con 0x7fb8c810d180 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.886+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fb8c0007500 con 0x7fb8c810d180 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8bffff700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb8b409ff50 0x7fb8b40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:44.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb8c00cba70 con 0x7fb8c810d180 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8be59c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb8b409ff50 0x7fb8b40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8bffff700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] conn(0x7fb8b40aaef0 0x7fb8b40ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8b40ada60 con 0x7fb8b40aaef0 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.888+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8c00cbde0 con 0x7fb8c810d180 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.889+0000 7fb8be59c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb8b409ff50 0x7fb8b40a2410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800b2e0 tx=0x7fb8b8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.889+0000 7fb8c659c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] conn(0x7fb8b40aaef0 0x7fb8b40ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:44.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.889+0000 7fb8c659c700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] conn(0x7fb8b40aaef0 0x7fb8b40ad390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:44.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.890+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== osd.11 v2:172.21.15.183:6824/1108166266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb8b40ada60 con 0x7fb8b40aaef0 2023-08-25T04:43:44.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.906+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 --> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8c80fe350 con 0x7fb8b40aaef0 2023-08-25T04:43:44.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.907+0000 7fb8bffff700 1 -- 172.21.15.145:0/3656492741 <== osd.11 v2:172.21.15.183:6824/1108166266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb8c80fe350 con 0x7fb8b40aaef0 2023-08-25T04:43:44.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.907+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] conn(0x7fb8b40aaef0 msgr2=0x7fb8b40ad390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.907+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] conn(0x7fb8b40aaef0 0x7fb8b40ad390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.907+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb8b409ff50 msgr2=0x7fb8b40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.907+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb8b409ff50 0x7fb8b40a2410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800b2e0 tx=0x7fb8b8005fd0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c810d180 msgr2=0x7fb8c8198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 --2- 172.21.15.145:0/3656492741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8c810d180 0x7fb8c8198ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000efc0 tx=0x7fb8c000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:43:44.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8c6d9d700 1 -- 172.21.15.145:0/3656492741 reap_dead start 2023-08-25T04:43:44.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 shutdown_connections 2023-08-25T04:43:44.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 >> 172.21.15.145:0/3656492741 conn(0x7fb8c80fc410 msgr2=0x7fb8c810f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:44.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 shutdown_connections 2023-08-25T04:43:44.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:43.908+0000 7fb8ccb55700 1 -- 172.21.15.145:0/3656492741 wait complete. 2023-08-25T04:43:44.202 INFO:teuthology.orchestra.run.smithi145.stdout:219043332136 2023-08-25T04:43:44.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-08-25T04:43:44.343 INFO:teuthology.orchestra.run.smithi145.stdout:163208757297 2023-08-25T04:43:44.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-08-25T04:43:44.709 INFO:teuthology.orchestra.run.smithi145.stdout:335007449111 2023-08-25T04:43:44.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-08-25T04:43:45.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:44 smithi145 ceph-mon[97681]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:45.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:44 smithi162 ceph-mon[130941]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:45.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:44 smithi183 ceph-mon[131414]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:45.181 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:45.248 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:45.360 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:45.557 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:45.616 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:45.633 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:46.348 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:46.772 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:46.973 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:46 smithi145 ceph-mon[97681]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:47.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:46 smithi162 ceph-mon[130941]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:47.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:46 smithi183 ceph-mon[131414]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:47.243 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:47.246 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:47.563 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:49.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:48 smithi145 ceph-mon[97681]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:49.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:48 smithi162 ceph-mon[130941]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:49.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:48 smithi183 ceph-mon[131414]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:49.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 -- 172.21.15.145:0/2233332140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 msgr2=0x7f71d810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:49.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 --2- 172.21.15.145:0/2233332140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d810f7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b0a0 tx=0x7f71d000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 -- 172.21.15.145:0/2233332140 shutdown_connections 2023-08-25T04:43:49.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 --2- 172.21.15.145:0/2233332140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d810f7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 --2- 172.21.15.145:0/2233332140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100340 0x7f71d8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 --2- 172.21.15.145:0/2233332140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100c60 0x7f71d810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.022+0000 7f71deb3c700 1 -- 172.21.15.145:0/2233332140 >> 172.21.15.145:0/2233332140 conn(0x7f71d8076920 msgr2=0x7f71d8076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.023+0000 7f71deb3c700 1 -- 172.21.15.145:0/2233332140 shutdown_connections 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.023+0000 7f71deb3c700 1 -- 172.21.15.145:0/2233332140 wait complete. 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 Processor -- start 2023-08-25T04:43:49.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 -- start start 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100340 0x7f71d8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100c60 0x7f71d81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819dd70 con 0x7f71d810d3b0 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819dee0 con 0x7f71d8100c60 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71deb3c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d819e0a0 con 0x7f71d8100340 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71dd0d9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:49.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71dd0d9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36912/0 (socket says 172.21.15.145:36912) 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71dd0d9700 1 -- 172.21.15.145:0/1363112124 learned_addr learned my addr 172.21.15.145:0/1363112124 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71d7fff700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100c60 0x7f71d81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.024+0000 7f71dc8d8700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100340 0x7f71d8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 -- 172.21.15.145:0/1363112124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100340 msgr2=0x7f71d8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100340 0x7f71d8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 -- 172.21.15.145:0/1363112124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100c60 msgr2=0x7f71d81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100c60 0x7f71d81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 -- 172.21.15.145:0/1363112124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d0009d00 con 0x7f71d810d3b0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71dd0d9700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009fd0 tx=0x7f71d00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000e040 con 0x7f71d810d3b0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71d001d470 con 0x7f71d810d3b0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d001b6c0 con 0x7f71d810d3b0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.025+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d819e320 con 0x7f71d810d3b0 2023-08-25T04:43:49.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.026+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d81a5a10 con 0x7f71d810d3b0 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.027+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d804e620 con 0x7f71d810d3b0 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.028+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f71d001d5e0 con 0x7f71d810d3b0 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.030+0000 7f71d5ffb700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 0x7f71c80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.030+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f71d00cc320 con 0x7f71d810d3b0 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.030+0000 7f71dc8d8700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 0x7f71c80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:49.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.031+0000 7f71dc8d8700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 0x7f71c80a2410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009c60 tx=0x7f71cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:49.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.034+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f71d0095de0 con 0x7f71d810d3b0 2023-08-25T04:43:49.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.278+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71d8112d30 con 0x7f71d810d3b0 2023-08-25T04:43:49.307 INFO:teuthology.orchestra.run.smithi145.stdout:111669149752 2023-08-25T04:43:49.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.282+0000 7f71d5ffb700 1 -- 172.21.15.145:0/1363112124 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71d0099400 con 0x7f71d810d3b0 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.287+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 msgr2=0x7f71c80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.287+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 0x7f71c80a2410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009c60 tx=0x7f71cc009260 comp rx=0 tx=0).stop 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.287+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 msgr2=0x7f71d819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.287+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009fd0 tx=0x7f71d00094a0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 shutdown_connections 2023-08-25T04:43:49.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f71c809ff50 0x7f71c80a2410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71d810d3b0 0x7f71d819d760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8100340 0x7f71d8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 --2- 172.21.15.145:0/1363112124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71d8100c60 0x7f71d81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.288+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 >> 172.21.15.145:0/1363112124 conn(0x7f71d8076920 msgr2=0x7f71d810e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.290+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 shutdown_connections 2023-08-25T04:43:49.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:49.290+0000 7f71deb3c700 1 -- 172.21.15.145:0/1363112124 wait complete. 2023-08-25T04:43:49.642 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:49.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:49 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1363112124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T04:43:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:49 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1363112124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T04:43:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:49 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1363112124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-25T04:43:50.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149750 got 111669149752 for osd.3 2023-08-25T04:43:50.456 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:50.817 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:50 smithi145 ceph-mon[97681]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:50.818 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:50 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:43:51.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:50 smithi162 ceph-mon[130941]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:51.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:50 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:43:51.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:50 smithi183 ceph-mon[131414]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-08-25T04:43:51.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:50 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:43:51.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.201+0000 7f16eaf55700 1 -- 172.21.15.145:0/4037661572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41038b0 msgr2=0x7f16e4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.201+0000 7f16eaf55700 1 --2- 172.21.15.145:0/4037661572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41038b0 0x7f16e4103d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f16d00098f0 tx=0x7f16d0009c00 comp rx=0 tx=0).stop 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 -- 172.21.15.145:0/4037661572 shutdown_connections 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 --2- 172.21.15.145:0/4037661572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 0x7f16e40688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 --2- 172.21.15.145:0/4037661572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41058e0 0x7f16e4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 --2- 172.21.15.145:0/4037661572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41038b0 0x7f16e4103d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 -- 172.21.15.145:0/4037661572 >> 172.21.15.145:0/4037661572 conn(0x7f16e4076920 msgr2=0x7f16e4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 -- 172.21.15.145:0/4037661572 shutdown_connections 2023-08-25T04:43:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.202+0000 7f16eaf55700 1 -- 172.21.15.145:0/4037661572 wait complete. 2023-08-25T04:43:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.204+0000 7f16eaf55700 1 Processor -- start 2023-08-25T04:43:51.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 -- start start 2023-08-25T04:43:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 0x7f16e419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41038b0 0x7f16e419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a2090 con 0x7f16e4068440 2023-08-25T04:43:51.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a2200 con 0x7f16e41058e0 2023-08-25T04:43:51.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.205+0000 7f16eaf55700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a23c0 con 0x7f16e41038b0 2023-08-25T04:43:51.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41038b0 0x7f16e419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:55166/0 (socket says 172.21.15.145:55166) 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 -- 172.21.15.145:0/3680584700 learned_addr learned my addr 172.21.15.145:0/3680584700 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e8cf1700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 0x7f16e419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 -- 172.21.15.145:0/3680584700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41038b0 msgr2=0x7f16e419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41038b0 0x7f16e419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 -- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 msgr2=0x7f16e419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 0x7f16e419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.206+0000 7f16e94f2700 1 -- 172.21.15.145:0/3680584700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009d00 con 0x7f16e41058e0 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16e94f2700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c340 tx=0x7f16dc00c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00e050 con 0x7f16e41058e0 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d0009520 con 0x7f16e41058e0 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc00f040 con 0x7f16e41058e0 2023-08-25T04:43:51.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007af0 con 0x7f16e41058e0 2023-08-25T04:43:51.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.207+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e41a9d30 con 0x7f16e41058e0 2023-08-25T04:43:51.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.209+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e404e620 con 0x7f16e41058e0 2023-08-25T04:43:51.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.211+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f16dc007c50 con 0x7f16e41058e0 2023-08-25T04:43:51.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.213+0000 7f16e1ffb700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 0x7f16d40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.213+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f16dc0cb250 con 0x7f16e41058e0 2023-08-25T04:43:51.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.214+0000 7f16e8cf1700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 0x7f16d40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.214+0000 7f16e8cf1700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 0x7f16d40a2490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16d8005950 tx=0x7f16d80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.217+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f16dc094ce0 con 0x7f16e41058e0 2023-08-25T04:43:51.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 -- 172.21.15.145:0/2051766138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 msgr2=0x7f3ec4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2051766138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4100dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009990 tx=0x7f3eb0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 -- 172.21.15.145:0/2051766138 shutdown_connections 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2051766138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2051766138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 0x7f3ec4106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2051766138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4100dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 -- 172.21.15.145:0/2051766138 >> 172.21.15.145:0/2051766138 conn(0x7f3ec40fc440 msgr2=0x7f3ec40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 -- 172.21.15.145:0/2051766138 shutdown_connections 2023-08-25T04:43:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.411+0000 7f3ecb053700 1 -- 172.21.15.145:0/2051766138 wait complete. 2023-08-25T04:43:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.412+0000 7f3ecb053700 1 Processor -- start 2023-08-25T04:43:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.412+0000 7f3ecb053700 1 -- start start 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 0x7f3ec40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a4ea0 con 0x7f3ec4106950 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a5010 con 0x7f3ec4101310 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ecb053700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41a51d0 con 0x7f3ec4100950 2023-08-25T04:43:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:55180/0 (socket says 172.21.15.145:55180) 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec95f0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 0x7f3ec40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 -- 172.21.15.145:0/2064788639 learned_addr learned my addr 172.21.15.145:0/2064788639 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec8def700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 -- 172.21.15.145:0/2064788639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 msgr2=0x7f3ec4072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 -- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 msgr2=0x7f3ec40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.413+0000 7f3ec3fff700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 0x7f3ec40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec3fff700 1 -- 172.21.15.145:0/2064788639 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00095f0 con 0x7f3ec4101310 2023-08-25T04:43:51.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec8def700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:51.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec3fff700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005910 tx=0x7f3eb0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb001d070 con 0x7f3ec4101310 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb0022470 con 0x7f3ec4101310 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec41a5450 con 0x7f3ec4101310 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0017650 con 0x7f3ec4101310 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.414+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec41a58c0 con 0x7f3ec4101310 2023-08-25T04:43:51.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.416+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec404e620 con 0x7f3ec4101310 2023-08-25T04:43:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.419+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3eb00225e0 con 0x7f3ec4101310 2023-08-25T04:43:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.420+0000 7f3ec1ffb700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 0x7f3eb40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.420+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3eb00cc740 con 0x7f3ec4101310 2023-08-25T04:43:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.421+0000 7f3ec8def700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 0x7f3eb40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.421+0000 7f3ec8def700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 0x7f3eb40a2490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009c90 tx=0x7f3eb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.425+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3eb005ce00 con 0x7f3ec4101310 2023-08-25T04:43:51.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.466+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16e4117110 con 0x7f16e41058e0 2023-08-25T04:43:51.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.467+0000 7f16e1ffb700 1 -- 172.21.15.145:0/3680584700 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16dc098300 con 0x7f16e41058e0 2023-08-25T04:43:51.471 INFO:teuthology.orchestra.run.smithi145.stdout:77309411391 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 msgr2=0x7f16d40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 0x7f16d40a2490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16d8005950 tx=0x7f16d80058e0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 msgr2=0x7f16e41a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c340 tx=0x7f16dc00c650 comp rx=0 tx=0).stop 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 shutdown_connections 2023-08-25T04:43:51.472 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f16d409ffd0 0x7f16d40a2490 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e41058e0 0x7f16e41a1a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16e4068440 0x7f16e419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 --2- 172.21.15.145:0/3680584700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16e41038b0 0x7f16e419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.471+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 >> 172.21.15.145:0/3680584700 conn(0x7f16e4076920 msgr2=0x7f16e40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.472+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 shutdown_connections 2023-08-25T04:43:51.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.472+0000 7f16eaf55700 1 -- 172.21.15.145:0/3680584700 wait complete. 2023-08-25T04:43:51.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/3625515708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74101300 msgr2=0x7fbc7410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/3625515708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74101300 0x7fbc7410d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009f90 tx=0x7fbc6c00b330 comp rx=0 tx=0).stop 2023-08-25T04:43:51.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/3625515708 shutdown_connections 2023-08-25T04:43:51.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/3625515708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74101300 0x7fbc7410d810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/3625515708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74106940 0x7fbc74106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/3625515708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc74100940 0x7fbc74100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/3625515708 >> 172.21.15.145:0/3625515708 conn(0x7fbc740fc410 msgr2=0x7fbc740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.706+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/3625515708 shutdown_connections 2023-08-25T04:43:51.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.707+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/3625515708 wait complete. 2023-08-25T04:43:51.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.708+0000 7fbc7c3cf700 1 Processor -- start 2023-08-25T04:43:51.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.708+0000 7fbc7c3cf700 1 -- start start 2023-08-25T04:43:51.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74100940 0x7fbc74072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc74106940 0x7fbc740736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc741a4f00 con 0x7fbc74101300 2023-08-25T04:43:51.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc741a5070 con 0x7fbc74106940 2023-08-25T04:43:51.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7c3cf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc741a5230 con 0x7fbc74100940 2023-08-25T04:43:51.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7996a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7996a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:36966/0 (socket says 172.21.15.145:36966) 2023-08-25T04:43:51.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.709+0000 7fbc7996a700 1 -- 172.21.15.145:0/1334986280 learned_addr learned my addr 172.21.15.145:0/1334986280 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:51.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 -- 172.21.15.145:0/1334986280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74100940 msgr2=0x7fbc74072c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T04:43:51.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74100940 0x7fbc74072c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 -- 172.21.15.145:0/1334986280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc74106940 msgr2=0x7fbc740736e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T04:43:51.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc74106940 0x7fbc740736e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 -- 172.21.15.145:0/1334986280 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009c40 con 0x7fbc74101300 2023-08-25T04:43:51.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.710+0000 7fbc7996a700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b550 tx=0x7fbc6400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.711+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011830 con 0x7fbc74101300 2023-08-25T04:43:51.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.711+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc64011e50 con 0x7fbc74101300 2023-08-25T04:43:51.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.711+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400f2e0 con 0x7fbc74101300 2023-08-25T04:43:51.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.712+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc741a5510 con 0x7fbc74101300 2023-08-25T04:43:51.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.712+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc741a59c0 con 0x7fbc74101300 2023-08-25T04:43:51.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.713+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7404e620 con 0x7fbc74101300 2023-08-25T04:43:51.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.715+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fbc64010580 con 0x7fbc74101300 2023-08-25T04:43:51.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.718+0000 7fbc6b7fe700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 0x7fbc600a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:51.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.718+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbc640caff0 con 0x7fbc74101300 2023-08-25T04:43:51.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.718+0000 7fbc7a16b700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 0x7fbc600a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:51.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.719+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3ec4110ba0 con 0x7f3ec4101310 2023-08-25T04:43:51.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.719+0000 7fbc7a16b700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 0x7fbc600a2490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005fd0 tx=0x7fbc70009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:51.721 INFO:teuthology.orchestra.run.smithi145.stdout:197568495662 2023-08-25T04:43:51.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.719+0000 7f3ec1ffb700 1 -- 172.21.15.145:0/2064788639 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3eb0096260 con 0x7f3ec4101310 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 msgr2=0x7f3eb40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 0x7f3eb40a2490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009c90 tx=0x7f3eb8009260 comp rx=0 tx=0).stop 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 msgr2=0x7f3ec4073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005910 tx=0x7f3eb0004710 comp rx=0 tx=0).stop 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 shutdown_connections 2023-08-25T04:43:51.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3eb409ffd0 0x7f3eb40a2490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ec4106950 0x7f3ec40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec4100950 0x7f3ec4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 --2- 172.21.15.145:0/2064788639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec4101310 0x7f3ec4073160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 >> 172.21.15.145:0/2064788639 conn(0x7f3ec40fc440 msgr2=0x7f3ec40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 shutdown_connections 2023-08-25T04:43:51.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.722+0000 7f3ecb053700 1 -- 172.21.15.145:0/2064788639 wait complete. 2023-08-25T04:43:51.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.726+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbc64094a20 con 0x7fbc74101300 2023-08-25T04:43:51.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.945+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbc74074150 con 0x7fbc74101300 2023-08-25T04:43:51.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.946+0000 7fbc6b7fe700 1 -- 172.21.15.145:0/1334986280 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc64098040 con 0x7fbc74101300 2023-08-25T04:43:51.947 INFO:teuthology.orchestra.run.smithi145.stdout:34359738440 2023-08-25T04:43:51.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.948+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 msgr2=0x7fbc600a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.948+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 0x7fbc600a2490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005fd0 tx=0x7fbc70009310 comp rx=0 tx=0).stop 2023-08-25T04:43:51.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.948+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 msgr2=0x7fbc74073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:51.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.948+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b550 tx=0x7fbc6400b910 comp rx=0 tx=0).stop 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 shutdown_connections 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbc6009ffd0 0x7fbc600a2490 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc74106940 0x7fbc740736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc74100940 0x7fbc74072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 --2- 172.21.15.145:0/1334986280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc74101300 0x7fbc74073160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:51.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 >> 172.21.15.145:0/1334986280 conn(0x7fbc740fc410 msgr2=0x7fbc74104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:51.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 shutdown_connections 2023-08-25T04:43:51.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:51.949+0000 7fbc7c3cf700 1 -- 172.21.15.145:0/1334986280 wait complete. 2023-08-25T04:43:51.971 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:51 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3680584700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T04:43:51.971 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:51 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2064788639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T04:43:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:51 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3680584700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T04:43:52.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:51 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2064788639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T04:43:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:51 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3680584700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-25T04:43:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:51 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2064788639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-25T04:43:52.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411388 got 77309411391 for osd.2 2023-08-25T04:43:52.842 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:53.059 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:52 smithi145 ceph-mon[97681]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:53.059 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1334986280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T04:43:53.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 -- 172.21.15.145:0/1847572009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 msgr2=0x7f4fd4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/1847572009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd4068cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00099c0 tx=0x7f4fd0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 -- 172.21.15.145:0/1847572009 shutdown_connections 2023-08-25T04:43:53.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/1847572009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105d50 0x7f4fd410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/1847572009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd4068cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/1847572009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105390 0x7f4fd4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 -- 172.21.15.145:0/1847572009 >> 172.21.15.145:0/1847572009 conn(0x7f4fd4076920 msgr2=0x7f4fd4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:53.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 -- 172.21.15.145:0/1847572009 shutdown_connections 2023-08-25T04:43:53.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.132+0000 7f4fdc53a700 1 -- 172.21.15.145:0/1847572009 wait complete. 2023-08-25T04:43:53.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.133+0000 7f4fdc53a700 1 Processor -- start 2023-08-25T04:43:53.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.133+0000 7f4fdc53a700 1 -- start start 2023-08-25T04:43:53.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 0x7f4fd419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105d50 0x7f4fd4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4199140 con 0x7f4fd40688e0 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41992b0 con 0x7f4fd4105390 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdc53a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4199470 con 0x7f4fd4105d50 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fda2d6700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fda2d6700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51514/0 (socket says 172.21.15.145:51514) 2023-08-25T04:43:53.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fda2d6700 1 -- 172.21.15.145:0/3351138182 learned_addr learned my addr 172.21.15.145:0/3351138182 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fd9ad5700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 0x7f4fd419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fdaad7700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105d50 0x7f4fd4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.134+0000 7f4fd9ad5700 1 -- 172.21.15.145:0/3351138182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105d50 msgr2=0x7f4fd4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fd9ad5700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105d50 0x7f4fd4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fd9ad5700 1 -- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 msgr2=0x7f4fd419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fd9ad5700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fd9ad5700 1 -- 172.21.15.145:0/3351138182 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd00095f0 con 0x7f4fd4105390 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fda2d6700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fd9ad5700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 0x7f4fd419ef90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400d950 tx=0x7f4fc400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400f890 con 0x7f4fd4105390 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd4199750 con 0x7f4fd4105390 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc4010460 con 0x7f4fd4105390 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400e610 con 0x7f4fd4105390 2023-08-25T04:43:53.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.135+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41a5a40 con 0x7f4fd4105390 2023-08-25T04:43:53.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.136+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd404e620 con 0x7f4fd4105390 2023-08-25T04:43:53.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.138+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4fc40108d0 con 0x7f4fd4105390 2023-08-25T04:43:53.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.139+0000 7f4fcb7fe700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 0x7f4fc00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.139+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4fc40cb5a0 con 0x7f4fd4105390 2023-08-25T04:43:53.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.140+0000 7f4fda2d6700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 0x7f4fc00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.140+0000 7f4fda2d6700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 0x7f4fc00a2410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0000c00 tx=0x7f4fd0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.143+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4fc4094fd0 con 0x7f4fd4105390 2023-08-25T04:43:53.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:52 smithi162 ceph-mon[130941]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1334986280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T04:43:53.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:52 smithi183 ceph-mon[131414]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:53.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1334986280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-25T04:43:53.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738438 got 34359738440 for osd.0 2023-08-25T04:43:53.258 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:53.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 -- 172.21.15.145:0/4011793523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc106940 msgr2=0x7f99fc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 --2- 172.21.15.145:0/4011793523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc106940 0x7f99fc106d20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009990 tx=0x7f99ec009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 -- 172.21.15.145:0/4011793523 shutdown_connections 2023-08-25T04:43:53.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 --2- 172.21.15.145:0/4011793523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc101300 0x7f99fc10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 --2- 172.21.15.145:0/4011793523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc106940 0x7f99fc106d20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 --2- 172.21.15.145:0/4011793523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc100940 0x7f99fc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 -- 172.21.15.145:0/4011793523 >> 172.21.15.145:0/4011793523 conn(0x7f99fc0fc410 msgr2=0x7f99fc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:53.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 -- 172.21.15.145:0/4011793523 shutdown_connections 2023-08-25T04:43:53.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.338+0000 7f9a01b71700 1 -- 172.21.15.145:0/4011793523 wait complete. 2023-08-25T04:43:53.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.339+0000 7f9a01b71700 1 Processor -- start 2023-08-25T04:43:53.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.339+0000 7f9a01b71700 1 -- start start 2023-08-25T04:43:53.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc100940 0x7f99fc19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc106940 0x7f99fc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc199110 con 0x7f99fc100940 2023-08-25T04:43:53.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc199280 con 0x7f99fc106940 2023-08-25T04:43:53.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f9a01b71700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc199440 con 0x7f99fc101300 2023-08-25T04:43:53.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34000/0 (socket says 172.21.15.145:34000) 2023-08-25T04:43:53.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc106940 0x7f99fc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 -- 172.21.15.145:0/1082205403 learned_addr learned my addr 172.21.15.145:0/1082205403 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:53.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99f37fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc100940 0x7f99fc19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 -- 172.21.15.145:0/1082205403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc106940 msgr2=0x7f99fc198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc106940 0x7f99fc198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 -- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc100940 msgr2=0x7f99fc19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc100940 0x7f99fc19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.340+0000 7f99fb7fe700 1 -- 172.21.15.145:0/1082205403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0095f0 con 0x7f99fc101300 2023-08-25T04:43:53.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f99fb7fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e9a0 tx=0x7f99e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4009790 con 0x7f99fc101300 2023-08-25T04:43:53.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e4010430 con 0x7f99fc101300 2023-08-25T04:43:53.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc199720 con 0x7f99fc101300 2023-08-25T04:43:53.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4018680 con 0x7f99fc101300 2023-08-25T04:43:53.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.341+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc1a5a10 con 0x7f99fc101300 2023-08-25T04:43:53.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.342+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc04e620 con 0x7f99fc101300 2023-08-25T04:43:53.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.344+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f99e40105a0 con 0x7f99fc101300 2023-08-25T04:43:53.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.346+0000 7f99f97fa700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 0x7f99e80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.346+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f99e4014070 con 0x7f99fc101300 2023-08-25T04:43:53.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.347+0000 7f99f37fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 0x7f99e80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.347+0000 7f99f37fe700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 0x7f99e80a2490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b2e0 tx=0x7f99ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.349+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f99e4051e00 con 0x7f99fc101300 2023-08-25T04:43:53.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.405+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4fd4110960 con 0x7f4fd4105390 2023-08-25T04:43:53.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.406+0000 7f4fcb7fe700 1 -- 172.21.15.145:0/3351138182 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4fc40985f0 con 0x7f4fd4105390 2023-08-25T04:43:53.407 INFO:teuthology.orchestra.run.smithi145.stdout:283467841568 2023-08-25T04:43:53.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 msgr2=0x7f4fc00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 0x7f4fc00a2410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0000c00 tx=0x7f4fd0005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 msgr2=0x7f4fd419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 0x7f4fd419ef90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400d950 tx=0x7f4fc400dc60 comp rx=0 tx=0).stop 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 shutdown_connections 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4fc009ff50 0x7f4fc00a2410 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd4105d50 0x7f4fd4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4fd40688e0 0x7f4fd419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 --2- 172.21.15.145:0/3351138182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fd4105390 0x7f4fd419ef90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.408+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 >> 172.21.15.145:0/3351138182 conn(0x7f4fd4076920 msgr2=0x7f4fd410ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:53.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.409+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 shutdown_connections 2023-08-25T04:43:53.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.409+0000 7f4fdc53a700 1 -- 172.21.15.145:0/3351138182 wait complete. 2023-08-25T04:43:53.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.605+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f99fc1109c0 con 0x7f99fc101300 2023-08-25T04:43:53.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.606+0000 7f99f97fa700 1 -- 172.21.15.145:0/1082205403 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99e4095100 con 0x7f99fc101300 2023-08-25T04:43:53.607 INFO:teuthology.orchestra.run.smithi145.stdout:163208757299 2023-08-25T04:43:53.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 msgr2=0x7f99e80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 0x7f99e80a2490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b2e0 tx=0x7f99ec005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 msgr2=0x7f99fc19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e9a0 tx=0x7f99e400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 shutdown_connections 2023-08-25T04:43:53.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f99e809ffd0 0x7f99e80a2490 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc106940 0x7f99fc198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99fc100940 0x7f99fc19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 --2- 172.21.15.145:0/1082205403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99fc101300 0x7f99fc19ef60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.608+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 >> 172.21.15.145:0/1082205403 conn(0x7f99fc0fc410 msgr2=0x7f99fc0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:53.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.609+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 shutdown_connections 2023-08-25T04:43:53.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.609+0000 7f9a01b71700 1 -- 172.21.15.145:0/1082205403 wait complete. 2023-08-25T04:43:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.785+0000 7fc173a59700 1 -- 172.21.15.145:0/1727969869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c0688e0 msgr2=0x7fc16c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.785+0000 7fc173a59700 1 --2- 172.21.15.145:0/1727969869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c0688e0 0x7fc16c068cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1680099c0 tx=0x7fc168009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 -- 172.21.15.145:0/1727969869 shutdown_connections 2023-08-25T04:43:53.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 --2- 172.21.15.145:0/1727969869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105d50 0x7fc16c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 --2- 172.21.15.145:0/1727969869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c0688e0 0x7fc16c068cc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 --2- 172.21.15.145:0/1727969869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c105390 0x7fc16c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 -- 172.21.15.145:0/1727969869 >> 172.21.15.145:0/1727969869 conn(0x7fc16c076920 msgr2=0x7fc16c076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 -- 172.21.15.145:0/1727969869 shutdown_connections 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.786+0000 7fc173a59700 1 -- 172.21.15.145:0/1727969869 wait complete. 2023-08-25T04:43:53.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 Processor -- start 2023-08-25T04:43:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 -- start start 2023-08-25T04:43:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c0688e0 0x7fc16c0ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c105d50 0x7fc16c1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c100700 con 0x7fc16c0688e0 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1a5200 con 0x7fc16c105390 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.787+0000 7fc173a59700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1a5370 con 0x7fc16c105d50 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:37354/0 (socket says 172.21.15.145:37354) 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 -- 172.21.15.145:0/686802011 learned_addr learned my addr 172.21.15.145:0/686802011 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:53.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc1717f5700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c0688e0 0x7fc16c0ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc171ff6700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c105d50 0x7fc16c1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 -- 172.21.15.145:0/686802011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c105d50 msgr2=0x7fc16c1a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c105d50 0x7fc16c1a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 -- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c0688e0 msgr2=0x7fc16c0ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c0688e0 0x7fc16c0ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 -- 172.21.15.145:0/686802011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009520 con 0x7fc16c105390 2023-08-25T04:43:53.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.788+0000 7fc170ff4700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00eb30 tx=0x7fc15c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.789+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c009880 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.789+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc15c0044e0 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.789+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1680095f0 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.789+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c003d80 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.789+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c1a59b0 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.790+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c04e620 con 0x7fc16c105390 2023-08-25T04:43:53.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.792+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc15c003f20 con 0x7fc16c105390 2023-08-25T04:43:53.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.793+0000 7fc1627fc700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 0x7fc1580a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:53.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.793+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc15c014070 con 0x7fc16c105390 2023-08-25T04:43:53.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.793+0000 7fc1717f5700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 0x7fc1580a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:53.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.794+0000 7fc1717f5700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 0x7fc1580a2490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc168000c00 tx=0x7fc168005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:53.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:53.797+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc15c095480 con 0x7fc16c105390 2023-08-25T04:43:54.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.007+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fc16c100e00 con 0x7fc16c105390 2023-08-25T04:43:54.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.008+0000 7fc1627fc700 1 -- 172.21.15.145:0/686802011 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc15c098aa0 con 0x7fc16c105390 2023-08-25T04:43:54.008 INFO:teuthology.orchestra.run.smithi145.stdout:249108103205 2023-08-25T04:43:54.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 msgr2=0x7fc1580a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:54.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 0x7fc1580a2490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc168000c00 tx=0x7fc168005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 msgr2=0x7fc16c0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00eb30 tx=0x7fc15c00ee40 comp rx=0 tx=0).stop 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 shutdown_connections 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc15809ffd0 0x7fc1580a2490 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16c105d50 0x7fc16c1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc16c0688e0 0x7fc16c0ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:54.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 --2- 172.21.15.145:0/686802011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc16c105390 0x7fc16c0ffe10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:54.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.010+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 >> 172.21.15.145:0/686802011 conn(0x7fc16c076920 msgr2=0x7fc16c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:54.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.011+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 shutdown_connections 2023-08-25T04:43:54.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.011+0000 7fc173a59700 1 -- 172.21.15.145:0/686802011 wait complete. 2023-08-25T04:43:54.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3351138182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1082205403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T04:43:54.579 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T04:43:54.580 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/686802011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T04:43:54.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841566 got 283467841568 for osd.9 2023-08-25T04:43:54.603 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:54.661 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3351138182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T04:43:54.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1082205403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T04:43:54.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T04:43:54.663 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/686802011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T04:43:54.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:43:54.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3351138182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-25T04:43:54.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1082205403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-25T04:43:54.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-08-25T04:43:54.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/686802011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-25T04:43:54.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757297 got 163208757299 for osd.5 2023-08-25T04:43:54.977 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:55.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/1430655047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4108b80 msgr2=0x7f5ba4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/1430655047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4108b80 0x7f5ba4108f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009990 tx=0x7f5b94009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/1430655047 shutdown_connections 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/1430655047 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba41034b0 0x7f5ba410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/1430655047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4108b80 0x7f5ba4108f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/1430655047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba4102af0 0x7f5ba4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/1430655047 >> 172.21.15.145:0/1430655047 conn(0x7f5ba40fe600 msgr2=0x7f5ba4100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/1430655047 shutdown_connections 2023-08-25T04:43:55.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.456+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/1430655047 wait complete. 2023-08-25T04:43:55.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.457+0000 7f5ba9eb1700 1 Processor -- start 2023-08-25T04:43:55.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 -- start start 2023-08-25T04:43:55.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba41034b0 0x7f5ba419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba4108b80 0x7f5ba4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.096 INFO:teuthology.orchestra.run.smithi145.stdout:304942678044 2023-08-25T04:43:55.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba41990b0 con 0x7f5ba4108b80 2023-08-25T04:43:55.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4199220 con 0x7f5ba41034b0 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba9eb1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba41993e0 con 0x7f5ba4102af0 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34040/0 (socket says 172.21.15.145:34040) 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba37fe700 1 -- 172.21.15.145:0/471431737 learned_addr learned my addr 172.21.15.145:0/471431737 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba3fff700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba4108b80 0x7f5ba4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.458+0000 7f5ba2ffd700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba41034b0 0x7f5ba419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 -- 172.21.15.145:0/471431737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba41034b0 msgr2=0x7f5ba419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba41034b0 0x7f5ba419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 -- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba4108b80 msgr2=0x7f5ba4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba4108b80 0x7f5ba4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 -- 172.21.15.145:0/471431737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940095f0 con 0x7f5ba4102af0 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba37fe700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005910 tx=0x7f5b94004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401d070 con 0x7f5ba4102af0 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b94022470 con 0x7f5ba4102af0 2023-08-25T04:43:55.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4199660 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94017650 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.459+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba41a5950 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.460+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba404e620 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.462+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5b940225e0 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.463+0000 7f5ba0ff9700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 0x7f5b8c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.463+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5b940cc6f0 con 0x7f5ba4102af0 2023-08-25T04:43:55.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.463+0000 7f5ba2ffd700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 0x7f5b8c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.464+0000 7f5ba2ffd700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 0x7f5b8c0a2490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4199c60 tx=0x7f5b98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.467+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5b9405ce00 con 0x7f5ba4102af0 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.682+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f5ba4112ad0 con 0x7f5ba4102af0 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.682+0000 7f5ba0ff9700 1 -- 172.21.15.145:0/471431737 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f5b940961b0 con 0x7f5ba4102af0 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 msgr2=0x7f5b8c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 0x7f5b8c0a2490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4199c60 tx=0x7f5b98009260 comp rx=0 tx=0).stop 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 msgr2=0x7f5ba419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005910 tx=0x7f5b94004710 comp rx=0 tx=0).stop 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 shutdown_connections 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5b8c09ffd0 0x7f5b8c0a2490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ba4108b80 0x7f5ba4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102af0 0x7f5ba419e960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 --2- 172.21.15.145:0/471431737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba41034b0 0x7f5ba419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 >> 172.21.15.145:0/471431737 conn(0x7f5ba40fe600 msgr2=0x7f5ba40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 shutdown_connections 2023-08-25T04:43:55.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:54.685+0000 7f5ba9eb1700 1 -- 172.21.15.145:0/471431737 wait complete. 2023-08-25T04:43:55.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495660 got 197568495662 for osd.6 2023-08-25T04:43:55.108 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 -- 172.21.15.145:0/219113202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c106940 msgr2=0x7f6c2c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 --2- 172.21.15.145:0/219113202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c106940 0x7f6c2c106d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009990 tx=0x7f6c1c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 -- 172.21.15.145:0/219113202 shutdown_connections 2023-08-25T04:43:55.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 --2- 172.21.15.145:0/219113202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c101300 0x7f6c2c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 --2- 172.21.15.145:0/219113202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c106940 0x7f6c2c106d20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 --2- 172.21.15.145:0/219113202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c100940 0x7f6c2c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 -- 172.21.15.145:0/219113202 >> 172.21.15.145:0/219113202 conn(0x7f6c2c0fc410 msgr2=0x7f6c2c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 -- 172.21.15.145:0/219113202 shutdown_connections 2023-08-25T04:43:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.320+0000 7f6c308a9700 1 -- 172.21.15.145:0/219113202 wait complete. 2023-08-25T04:43:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 Processor -- start 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 -- start start 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c100940 0x7f6c2c072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 0x7f6c2c073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 0x7f6c2c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1a4ec0 con 0x7f6c2c106940 2023-08-25T04:43:55.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1a5030 con 0x7f6c2c101300 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.321+0000 7f6c308a9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1a51f0 con 0x7f6c2c100940 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c29d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 0x7f6c2c073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 0x7f6c2c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c29d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 0x7f6c2c073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:37404/0 (socket says 172.21.15.145:37404) 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c29d9b700 1 -- 172.21.15.145:0/2277972206 learned_addr learned my addr 172.21.15.145:0/2277972206 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:55.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2a59c700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c100940 0x7f6c2c072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 -- 172.21.15.145:0/2277972206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c100940 msgr2=0x7f6c2c072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c100940 0x7f6c2c072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 -- 172.21.15.145:0/2277972206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 msgr2=0x7f6c2c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 0x7f6c2c073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.322+0000 7f6c2ad9d700 1 -- 172.21.15.145:0/2277972206 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0095f0 con 0x7f6c2c106940 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c2ad9d700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 0x7f6c2c0736e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400efc0 tx=0x7f6c2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24018070 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c2400f040 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c1a5470 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400cd40 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.323+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c1a5920 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.324+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c04e620 con 0x7f6c2c106940 2023-08-25T04:43:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.326+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f6c24003a00 con 0x7f6c2c106940 2023-08-25T04:43:55.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.328+0000 7f6c1b7fe700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 0x7f6c140a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.328+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6c240cb0f0 con 0x7f6c2c106940 2023-08-25T04:43:55.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.328+0000 7f6c2a59c700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 0x7f6c140a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.329+0000 7f6c2a59c700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 0x7f6c140a2410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b2e0 tx=0x7f6c1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.331+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6c24094bb0 con 0x7f6c2c106940 2023-08-25T04:43:55.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 -- 172.21.15.145:0/754400958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105390 msgr2=0x7fdd90105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 --2- 172.21.15.145:0/754400958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105390 0x7fdd90105810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd840099c0 tx=0x7fdd84009cd0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 -- 172.21.15.145:0/754400958 shutdown_connections 2023-08-25T04:43:55.517 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 --2- 172.21.15.145:0/754400958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd90105d50 0x7fdd9010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 --2- 172.21.15.145:0/754400958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd900688e0 0x7fdd90068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 --2- 172.21.15.145:0/754400958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105390 0x7fdd90105810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.332+0000 7fdd95793700 1 -- 172.21.15.145:0/754400958 >> 172.21.15.145:0/754400958 conn(0x7fdd90076920 msgr2=0x7fdd90076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.333+0000 7fdd95793700 1 -- 172.21.15.145:0/754400958 shutdown_connections 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.333+0000 7fdd95793700 1 -- 172.21.15.145:0/754400958 wait complete. 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 Processor -- start 2023-08-25T04:43:55.518 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 -- start start 2023-08-25T04:43:55.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd900688e0 0x7fdd9019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 0x7fdd90198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90199150 con 0x7fdd90105d50 2023-08-25T04:43:55.519 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90199290 con 0x7fdd900688e0 2023-08-25T04:43:55.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd95793700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90199450 con 0x7fdd90105390 2023-08-25T04:43:55.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd8e7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd8f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 0x7fdd90198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.334+0000 7fdd8e7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34078/0 (socket says 172.21.15.145:34078) 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 -- 172.21.15.145:0/4224275415 learned_addr learned my addr 172.21.15.145:0/4224275415 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8effd700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd900688e0 0x7fdd9019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 -- 172.21.15.145:0/4224275415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd900688e0 msgr2=0x7fdd9019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd900688e0 0x7fdd9019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 -- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 msgr2=0x7fdd90198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 0x7fdd90198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 -- 172.21.15.145:0/4224275415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840095f0 con 0x7fdd90105390 2023-08-25T04:43:55.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8f7fe700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 0x7fdd90198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:55.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.335+0000 7fdd8e7fc700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098e0 tx=0x7fdd8400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.336+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401d070 con 0x7fdd90105390 2023-08-25T04:43:55.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.336+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd84005030 con 0x7fdd90105390 2023-08-25T04:43:55.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.336+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd901996d0 con 0x7fdd90105390 2023-08-25T04:43:55.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.336+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84004a00 con 0x7fdd90105390 2023-08-25T04:43:55.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.336+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd901a5ab0 con 0x7fdd90105390 2023-08-25T04:43:55.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.337+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9004e620 con 0x7fdd90105390 2023-08-25T04:43:55.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.341+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fdd840051a0 con 0x7fdd90105390 2023-08-25T04:43:55.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.342+0000 7fdd77fff700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 0x7fdd780a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.342+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdd840cc520 con 0x7fdd90105390 2023-08-25T04:43:55.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.342+0000 7fdd8effd700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 0x7fdd780a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.343+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fdd84099a70 con 0x7fdd90105390 2023-08-25T04:43:55.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.343+0000 7fdd8effd700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 0x7fdd780a2490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009c60 tx=0x7fdd80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.561 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:55 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/471431737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T04:43:55.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.560+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdd901108e0 con 0x7fdd90105390 2023-08-25T04:43:55.563 INFO:teuthology.orchestra.run.smithi145.stdout:219043332138 2023-08-25T04:43:55.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.561+0000 7fdd77fff700 1 -- 172.21.15.145:0/4224275415 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd84099600 con 0x7fdd90105390 2023-08-25T04:43:55.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.561+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6c2c110b70 con 0x7f6c2c106940 2023-08-25T04:43:55.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.562+0000 7f6c1b7fe700 1 -- 172.21.15.145:0/2277972206 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c240981d0 con 0x7f6c2c106940 2023-08-25T04:43:55.565 INFO:teuthology.orchestra.run.smithi145.stdout:141733920823 2023-08-25T04:43:55.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 msgr2=0x7fdd780a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 0x7fdd780a2490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009c60 tx=0x7fdd80009260 comp rx=0 tx=0).stop 2023-08-25T04:43:55.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 msgr2=0x7fdd9019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098e0 tx=0x7fdd8400f690 comp rx=0 tx=0).stop 2023-08-25T04:43:55.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 shutdown_connections 2023-08-25T04:43:55.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdd7809ffd0 0x7fdd780a2490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd90105d50 0x7fdd90198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd900688e0 0x7fdd9019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 --2- 172.21.15.145:0/4224275415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd90105390 0x7fdd9019ef50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 >> 172.21.15.145:0/4224275415 conn(0x7fdd90076920 msgr2=0x7fdd9010bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 shutdown_connections 2023-08-25T04:43:55.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.564+0000 7fdd95793700 1 -- 172.21.15.145:0/4224275415 wait complete. 2023-08-25T04:43:55.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 msgr2=0x7f6c140a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 0x7f6c140a2410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b2e0 tx=0x7f6c1c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 msgr2=0x7f6c2c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 0x7f6c2c0736e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400efc0 tx=0x7f6c2400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 shutdown_connections 2023-08-25T04:43:55.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f6c1409ff50 0x7f6c140a2410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c2c106940 0x7f6c2c0736e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c2c100940 0x7f6c2c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 --2- 172.21.15.145:0/2277972206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c2c101300 0x7f6c2c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.565+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 >> 172.21.15.145:0/2277972206 conn(0x7f6c2c0fc410 msgr2=0x7f6c2c0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.566+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 shutdown_connections 2023-08-25T04:43:55.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.566+0000 7f6c308a9700 1 -- 172.21.15.145:0/2277972206 wait complete. 2023-08-25T04:43:55.752 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678042 got 304942678044 for osd.10 2023-08-25T04:43:55.753 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:55.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 -- 172.21.15.145:0/1221379057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 msgr2=0x7f4c54106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 --2- 172.21.15.145:0/1221379057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c54106d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c440098f0 tx=0x7f4c44009c00 comp rx=0 tx=0).stop 2023-08-25T04:43:55.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 -- 172.21.15.145:0/1221379057 shutdown_connections 2023-08-25T04:43:55.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 --2- 172.21.15.145:0/1221379057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54101300 0x7f4c5410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 --2- 172.21.15.145:0/1221379057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c54106d20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 --2- 172.21.15.145:0/1221379057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54100940 0x7f4c54100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 -- 172.21.15.145:0/1221379057 >> 172.21.15.145:0/1221379057 conn(0x7f4c540fc410 msgr2=0x7f4c540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:55.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 -- 172.21.15.145:0/1221379057 shutdown_connections 2023-08-25T04:43:55.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.901+0000 7f4c597e8700 1 -- 172.21.15.145:0/1221379057 wait complete. 2023-08-25T04:43:55.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.902+0000 7f4c597e8700 1 Processor -- start 2023-08-25T04:43:55.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.902+0000 7f4c597e8700 1 -- start start 2023-08-25T04:43:55.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 0x7f4c5419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.909 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c541968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.909 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54196f60 con 0x7f4c54106940 2023-08-25T04:43:55.909 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541970d0 con 0x7f4c54101300 2023-08-25T04:43:55.909 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c597e8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54197290 con 0x7f4c54100940 2023-08-25T04:43:55.910 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.910 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c541968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:37446/0 (socket says 172.21.15.145:37446) 2023-08-25T04:43:55.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 -- 172.21.15.145:0/2380967186 learned_addr learned my addr 172.21.15.145:0/2380967186 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:55.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c52ffd700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 0x7f4c5419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 -- 172.21.15.145:0/2380967186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 msgr2=0x7f4c5419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 0x7f4c5419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 -- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 msgr2=0x7f4c541968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:55.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c541968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:55.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c527fc700 1 -- 172.21.15.145:0/2380967186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009d00 con 0x7f4c54101300 2023-08-25T04:43:55.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c52ffd700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 0x7f4c5419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:43:55.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.903+0000 7f4c537fe700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c541968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c527fc700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9a0 tx=0x7f4c4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48009790 con 0x7f4c54101300 2023-08-25T04:43:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c44009520 con 0x7f4c54101300 2023-08-25T04:43:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c4800cbd0 con 0x7f4c54101300 2023-08-25T04:43:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48018680 con 0x7f4c54101300 2023-08-25T04:43:55.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.904+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c541a3690 con 0x7f4c54101300 2023-08-25T04:43:55.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.905+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5404e620 con 0x7f4c54101300 2023-08-25T04:43:55.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:55 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/471431737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T04:43:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.907+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4c4800cd40 con 0x7f4c54101300 2023-08-25T04:43:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.908+0000 7f4c3bfff700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 0x7f4c3c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.909+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4c48014070 con 0x7f4c54101300 2023-08-25T04:43:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.909+0000 7f4c52ffd700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 0x7f4c3c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.909+0000 7f4c52ffd700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 0x7f4c3c0a2410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c440098c0 tx=0x7f4c44019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:55.911+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4c480952f0 con 0x7f4c54101300 2023-08-25T04:43:55.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:55 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/471431737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-25T04:43:56.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.140+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c540008d0 con 0x7f4c54101300 2023-08-25T04:43:56.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.140+0000 7f4c3bfff700 1 -- 172.21.15.145:0/2380967186 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c48098910 con 0x7f4c54101300 2023-08-25T04:43:56.143 INFO:teuthology.orchestra.run.smithi145.stdout:55834574915 2023-08-25T04:43:56.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 msgr2=0x7f4c3c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 0x7f4c3c0a2410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c440098c0 tx=0x7f4c44019040 comp rx=0 tx=0).stop 2023-08-25T04:43:56.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 msgr2=0x7f4c5419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9a0 tx=0x7f4c4800ecb0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 shutdown_connections 2023-08-25T04:43:56.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4c3c09ff50 0x7f4c3c0a2410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c54106940 0x7f4c541968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c54100940 0x7f4c5419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 --2- 172.21.15.145:0/2380967186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c54101300 0x7f4c5419cdb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 >> 172.21.15.145:0/2380967186 conn(0x7f4c540fc410 msgr2=0x7f4c540fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:56.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 shutdown_connections 2023-08-25T04:43:56.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.143+0000 7f4c597e8700 1 -- 172.21.15.145:0/2380967186 wait complete. 2023-08-25T04:43:56.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920820 got 141733920823 for osd.4 2023-08-25T04:43:56.302 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:56.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 -- 172.21.15.145:0/93164486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4106940 msgr2=0x7fcab4106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 --2- 172.21.15.145:0/93164486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4106940 0x7fcab4106d20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009990 tx=0x7fcab0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 -- 172.21.15.145:0/93164486 shutdown_connections 2023-08-25T04:43:56.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 --2- 172.21.15.145:0/93164486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4101300 0x7fcab410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 --2- 172.21.15.145:0/93164486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4106940 0x7fcab4106d20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 --2- 172.21.15.145:0/93164486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4100940 0x7fcab4100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 -- 172.21.15.145:0/93164486 >> 172.21.15.145:0/93164486 conn(0x7fcab40fc410 msgr2=0x7fcab40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:56.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 -- 172.21.15.145:0/93164486 shutdown_connections 2023-08-25T04:43:56.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.622+0000 7fcabc215700 1 -- 172.21.15.145:0/93164486 wait complete. 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.623+0000 7fcabc215700 1 Processor -- start 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.623+0000 7fcabc215700 1 -- start start 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.623+0000 7fcabc215700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.623+0000 7fcabc215700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4101300 0x7fcab419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.623+0000 7fcabc215700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 0x7fcab4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:56.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcabc215700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4199100 con 0x7fcab4106940 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcabc215700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4199270 con 0x7fcab4100940 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcabc215700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4199430 con 0x7fcab4101300 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcaba7b2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 0x7fcab4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:37460/0 (socket says 172.21.15.145:37460) 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 -- 172.21.15.145:0/3518212751 learned_addr learned my addr 172.21.15.145:0/3518212751 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:56.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab97b0700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4101300 0x7fcab419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 -- 172.21.15.145:0/3518212751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4101300 msgr2=0x7fcab419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4101300 0x7fcab419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 -- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 msgr2=0x7fcab4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 0x7fcab4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcab9fb1700 1 -- 172.21.15.145:0/3518212751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00095f0 con 0x7fcab4100940 2023-08-25T04:43:56.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.624+0000 7fcaba7b2700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 0x7fcab4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcab9fb1700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005910 tx=0x7fcab0005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab001d070 con 0x7fcab4100940 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab41996b0 con 0x7fcab4100940 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab0022470 con 0x7fcab4100940 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0017650 con 0x7fcab4100940 2023-08-25T04:43:56.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.625+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab41a5a00 con 0x7fcab4100940 2023-08-25T04:43:56.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.626+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab404e620 con 0x7fcab4100940 2023-08-25T04:43:56.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.628+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcab00225e0 con 0x7fcab4100940 2023-08-25T04:43:56.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.629+0000 7fcaaaffd700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 0x7fcaa00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:56.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.629+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcab00cc9a0 con 0x7fcab4100940 2023-08-25T04:43:56.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.629+0000 7fcab97b0700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 0x7fcaa00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:56.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.630+0000 7fcab97b0700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 0x7fcaa00a2410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcab4105f10 tx=0x7fcaa4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:56.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.631+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcab0096460 con 0x7fcab4100940 2023-08-25T04:43:56.666 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332136 got 219043332138 for osd.7 2023-08-25T04:43:56.666 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:56.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4224275415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T04:43:56.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2277972206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T04:43:56.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:56 smithi145 ceph-mon[97681]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-08-25T04:43:56.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2380967186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T04:43:56.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.843+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fcab4110950 con 0x7fcab4100940 2023-08-25T04:43:56.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.844+0000 7fcaaaffd700 1 -- 172.21.15.145:0/3518212751 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fcab0027020 con 0x7fcab4100940 2023-08-25T04:43:56.846 INFO:teuthology.orchestra.run.smithi145.stdout:335007449114 2023-08-25T04:43:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 msgr2=0x7fcaa00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 0x7fcaa00a2410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcab4105f10 tx=0x7fcaa4009260 comp rx=0 tx=0).stop 2023-08-25T04:43:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 msgr2=0x7fcab419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005910 tx=0x7fcab0005720 comp rx=0 tx=0).stop 2023-08-25T04:43:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 shutdown_connections 2023-08-25T04:43:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcaa009ff50 0x7fcaa00a2410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcab4106940 0x7fcab4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcab4100940 0x7fcab419ea10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 --2- 172.21.15.145:0/3518212751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab4101300 0x7fcab419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 >> 172.21.15.145:0/3518212751 conn(0x7fcab40fc410 msgr2=0x7fcab410bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 shutdown_connections 2023-08-25T04:43:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:56.847+0000 7fcabc215700 1 -- 172.21.15.145:0/3518212751 wait complete. 2023-08-25T04:43:56.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103203 got 249108103205 for osd.8 2023-08-25T04:43:56.891 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:56.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4224275415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T04:43:56.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2277972206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T04:43:56.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:56 smithi162 ceph-mon[130941]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-08-25T04:43:56.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2380967186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T04:43:56.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4224275415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-25T04:43:56.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2277972206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-25T04:43:56.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:56 smithi183 ceph-mon[131414]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-08-25T04:43:56.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2380967186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-25T04:43:57.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574915 for osd.1 2023-08-25T04:43:57.318 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:57.406 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449111 got 335007449114 for osd.11 2023-08-25T04:43:57.407 DEBUG:teuthology.parallel:result is None 2023-08-25T04:43:57.407 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-08-25T04:43:57.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph pg dump --format=json 2023-08-25T04:43:57.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:57 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3518212751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T04:43:57.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:57 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3518212751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T04:43:57.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:57 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3518212751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-25T04:43:58.245 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:43:58.818 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:43:58 smithi145 ceph-mon[97681]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:58.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:43:58 smithi162 ceph-mon[130941]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:58.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:43:58 smithi183 ceph-mon[131414]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:43:59.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/2430140990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4106940 msgr2=0x7fd7f4106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:59.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/2430140990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4106940 0x7fd7f4106d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009990 tx=0x7fd7e4009ca0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/2430140990 shutdown_connections 2023-08-25T04:43:59.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/2430140990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 0x7fd7f410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/2430140990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4106940 0x7fd7f4106d20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/2430140990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4100940 0x7fd7f4100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/2430140990 >> 172.21.15.145:0/2430140990 conn(0x7fd7f40fc410 msgr2=0x7fd7f40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:59.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.720+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/2430140990 shutdown_connections 2023-08-25T04:43:59.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.721+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/2430140990 wait complete. 2023-08-25T04:43:59.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.721+0000 7fd7fa8e4700 1 Processor -- start 2023-08-25T04:43:59.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 -- start start 2023-08-25T04:43:59.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:59.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 0x7fd7f4073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:59.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4106940 0x7fd7f40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:59.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a4f10 con 0x7fd7f4100940 2023-08-25T04:43:59.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a5080 con 0x7fd7f4106940 2023-08-25T04:43:59.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7fa8e4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a5240 con 0x7fd7f4101300 2023-08-25T04:43:59.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7ebfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:59.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7ebfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51644/0 (socket says 172.21.15.145:51644) 2023-08-25T04:43:59.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7ebfff700 1 -- 172.21.15.145:0/3615212362 learned_addr learned my addr 172.21.15.145:0/3615212362 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:43:59.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.722+0000 7fd7f3fff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 0x7fd7f4073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:59.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7f8e81700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4106940 0x7fd7f40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:59.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 -- 172.21.15.145:0/3615212362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 msgr2=0x7fd7f4073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:59.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 0x7fd7f4073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 -- 172.21.15.145:0/3615212362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4106940 msgr2=0x7fd7f40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:59.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4106940 0x7fd7f40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 -- 172.21.15.145:0/3615212362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40095f0 con 0x7fd7f4100940 2023-08-25T04:43:59.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7ebfff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4005910 tx=0x7fd7e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:59.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e401d070 con 0x7fd7f4100940 2023-08-25T04:43:59.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f41a54c0 con 0x7fd7f4100940 2023-08-25T04:43:59.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7e4022470 con 0x7fd7f4100940 2023-08-25T04:43:59.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.723+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4017650 con 0x7fd7f4100940 2023-08-25T04:43:59.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.724+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f41a5930 con 0x7fd7f4100940 2023-08-25T04:43:59.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.724+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f404e620 con 0x7fd7f4100940 2023-08-25T04:43:59.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.726+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd7e400fad0 con 0x7fd7f4100940 2023-08-25T04:43:59.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.728+0000 7fd7f1ffb700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 0x7fd7e00a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:43:59.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.728+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd7e40cc9a0 con 0x7fd7f4100940 2023-08-25T04:43:59.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.728+0000 7fd7f3fff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 0x7fd7e00a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:43:59.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.729+0000 7fd7f3fff700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 0x7fd7e00a2490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4074410 tx=0x7fd7dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:43:59.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.732+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd7e4096460 con 0x7fd7f4100940 2023-08-25T04:43:59.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.946+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7f4110a30 con 0x7fd7e009ffd0 2023-08-25T04:43:59.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.961+0000 7fd7f1ffb700 1 -- 172.21.15.145:0/3615212362 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426978 (secure 0 0 0) 0x7fd7f4110a30 con 0x7fd7e009ffd0 2023-08-25T04:43:59.968 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:43:59.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 msgr2=0x7fd7e00a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:59.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 0x7fd7e00a2490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4074410 tx=0x7fd7dc009260 comp rx=0 tx=0).stop 2023-08-25T04:43:59.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 msgr2=0x7fd7f4072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4005910 tx=0x7fd7e4005740 comp rx=0 tx=0).stop 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 shutdown_connections 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd7e009ffd0 0x7fd7e00a2490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7f4106940 0x7fd7f40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7f4100940 0x7fd7f4072c20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 --2- 172.21.15.145:0/3615212362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f4101300 0x7fd7f4073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:43:59.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.966+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 >> 172.21.15.145:0/3615212362 conn(0x7fd7f40fc410 msgr2=0x7fd7f40fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:43:59.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.967+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 shutdown_connections 2023-08-25T04:43:59.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:43:59.967+0000 7fd7fa8e4700 1 -- 172.21.15.145:0/3615212362 wait complete. 2023-08-25T04:43:59.972 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:44:00.394 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-08-25T04:43:59.701296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":412,"num_read_kb":271,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335924,"kb_used_data":11652,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605900,"statfs":{"total":1151940427776,"available":1151596441600,"internally_reserved":0,"allocated":11931648,"data_stored":7585297,"data_compressed":318375,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003072"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765465+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:43:34.765465+0000","last_peered":"2023-08-25T04:43:34.765465+0000","last_clean":"2023-08-25T04:43:34.765465+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:43:34.765465+0000","last_undegraded":"2023-08-25T04:43:34.765465+0000","last_fullsized":"2023-08-25T04:43:34.765465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:59.835651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764768+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:43:34.764768+0000","last_peered":"2023-08-25T04:43:34.764768+0000","last_clean":"2023-08-25T04:43:34.764768+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:43:34.764768+0000","last_undegraded":"2023-08-25T04:43:34.764768+0000","last_fullsized":"2023-08-25T04:43:34.764768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:56:09.848052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766305+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:43:34.766305+0000","last_peered":"2023-08-25T04:43:34.766305+0000","last_clean":"2023-08-25T04:43:34.766305+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:43:34.766305+0000","last_undegraded":"2023-08-25T04:43:34.766305+0000","last_fullsized":"2023-08-25T04:43:34.766305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:56.683325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762343+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:43:34.762343+0000","last_peered":"2023-08-25T04:43:34.762343+0000","last_clean":"2023-08-25T04:43:34.762343+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:43:34.762343+0000","last_undegraded":"2023-08-25T04:43:34.762343+0000","last_fullsized":"2023-08-25T04:43:34.762343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:57:35.343381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766096+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:43:34.766096+0000","last_peered":"2023-08-25T04:43:34.766096+0000","last_clean":"2023-08-25T04:43:34.766096+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:43:34.766096+0000","last_undegraded":"2023-08-25T04:43:34.766096+0000","last_fullsized":"2023-08-25T04:43:34.766096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:53:26.027535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764535+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:43:34.764535+0000","last_peered":"2023-08-25T04:43:34.764535+0000","last_clean":"2023-08-25T04:43:34.764535+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:43:34.764535+0000","last_undegraded":"2023-08-25T04:43:34.764535+0000","last_fullsized":"2023-08-25T04:43:34.764535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:00.744928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922718+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:43:34.922718+0000","last_peered":"2023-08-25T04:43:34.922718+0000","last_clean":"2023-08-25T04:43:34.922718+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:43:34.922718+0000","last_undegraded":"2023-08-25T04:43:34.922718+0000","last_fullsized":"2023-08-25T04:43:34.922718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:47:46.522466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765008+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:43:34.765008+0000","last_peered":"2023-08-25T04:43:34.765008+0000","last_clean":"2023-08-25T04:43:34.765008+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:43:34.765008+0000","last_undegraded":"2023-08-25T04:43:34.765008+0000","last_fullsized":"2023-08-25T04:43:34.765008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:08:38.855500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099337+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:43:35.099337+0000","last_peered":"2023-08-25T04:43:35.099337+0000","last_clean":"2023-08-25T04:43:35.099337+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:43:35.099337+0000","last_undegraded":"2023-08-25T04:43:35.099337+0000","last_fullsized":"2023-08-25T04:43:35.099337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:36.964244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765117+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:43:34.765117+0000","last_peered":"2023-08-25T04:43:34.765117+0000","last_clean":"2023-08-25T04:43:34.765117+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:43:34.765117+0000","last_undegraded":"2023-08-25T04:43:34.765117+0000","last_fullsized":"2023-08-25T04:43:34.765117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:00.051214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923189+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:43:34.923189+0000","last_peered":"2023-08-25T04:43:34.923189+0000","last_clean":"2023-08-25T04:43:34.923189+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:43:34.923189+0000","last_undegraded":"2023-08-25T04:43:34.923189+0000","last_fullsized":"2023-08-25T04:43:34.923189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:24.087589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762199+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:43:34.762199+0000","last_peered":"2023-08-25T04:43:34.762199+0000","last_clean":"2023-08-25T04:43:34.762199+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:43:34.762199+0000","last_undegraded":"2023-08-25T04:43:34.762199+0000","last_fullsized":"2023-08-25T04:43:34.762199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:23.029577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099138+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:43:35.099138+0000","last_peered":"2023-08-25T04:43:35.099138+0000","last_clean":"2023-08-25T04:43:35.099138+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:43:35.099138+0000","last_undegraded":"2023-08-25T04:43:35.099138+0000","last_fullsized":"2023-08-25T04:43:35.099138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:53.794589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764677+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:43:34.764677+0000","last_peered":"2023-08-25T04:43:34.764677+0000","last_clean":"2023-08-25T04:43:34.764677+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:43:34.764677+0000","last_undegraded":"2023-08-25T04:43:34.764677+0000","last_fullsized":"2023-08-25T04:43:34.764677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:07.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764876+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:43:34.764876+0000","last_peered":"2023-08-25T04:43:34.764876+0000","last_clean":"2023-08-25T04:43:34.764876+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:43:34.764876+0000","last_undegraded":"2023-08-25T04:43:34.764876+0000","last_fullsized":"2023-08-25T04:43:34.764876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:15:49.317438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098418+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:43:35.098418+0000","last_peered":"2023-08-25T04:43:35.098418+0000","last_clean":"2023-08-25T04:43:35.098418+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:43:35.098418+0000","last_undegraded":"2023-08-25T04:43:35.098418+0000","last_fullsized":"2023-08-25T04:43:35.098418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:53.277068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765316+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:43:34.765316+0000","last_peered":"2023-08-25T04:43:34.765316+0000","last_clean":"2023-08-25T04:43:34.765316+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:43:34.765316+0000","last_undegraded":"2023-08-25T04:43:34.765316+0000","last_fullsized":"2023-08-25T04:43:34.765316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:04.508809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761810+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:43:34.761810+0000","last_peered":"2023-08-25T04:43:34.761810+0000","last_clean":"2023-08-25T04:43:34.761810+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:43:34.761810+0000","last_undegraded":"2023-08-25T04:43:34.761810+0000","last_fullsized":"2023-08-25T04:43:34.761810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:10:02.382799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770010+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:43:34.770010+0000","last_peered":"2023-08-25T04:43:34.770010+0000","last_clean":"2023-08-25T04:43:34.770010+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:43:34.770010+0000","last_undegraded":"2023-08-25T04:43:34.770010+0000","last_fullsized":"2023-08-25T04:43:34.770010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:00:34.306505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769760+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:43:34.769760+0000","last_peered":"2023-08-25T04:43:34.769760+0000","last_clean":"2023-08-25T04:43:34.769760+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:43:34.769760+0000","last_undegraded":"2023-08-25T04:43:34.769760+0000","last_fullsized":"2023-08-25T04:43:34.769760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:23.911843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761620+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:43:34.761620+0000","last_peered":"2023-08-25T04:43:34.761620+0000","last_clean":"2023-08-25T04:43:34.761620+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:43:34.761620+0000","last_undegraded":"2023-08-25T04:43:34.761620+0000","last_fullsized":"2023-08-25T04:43:34.761620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:30.639568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765547+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:43:34.765547+0000","last_peered":"2023-08-25T04:43:34.765547+0000","last_clean":"2023-08-25T04:43:34.765547+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:43:34.765547+0000","last_undegraded":"2023-08-25T04:43:34.765547+0000","last_fullsized":"2023-08-25T04:43:34.765547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:24.659622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100513+0000","last_change":"2023-08-25T04:43:35.100513+0000","last_active":"2023-08-25T04:43:35.100513+0000","last_peered":"2023-08-25T04:43:35.100513+0000","last_clean":"2023-08-25T04:43:35.100513+0000","last_became_active":"2023-08-25T04:43:35.100067+0000","last_became_peered":"2023-08-25T04:43:35.100067+0000","last_unstale":"2023-08-25T04:43:35.100513+0000","last_undegraded":"2023-08-25T04:43:35.100513+0000","last_fullsized":"2023-08-25T04:43:35.100513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:23:03.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765693+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:43:34.765693+0000","last_peered":"2023-08-25T04:43:34.765693+0000","last_clean":"2023-08-25T04:43:34.765693+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:43:34.765693+0000","last_undegraded":"2023-08-25T04:43:34.765693+0000","last_fullsized":"2023-08-25T04:43:34.765693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:38:39.362078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765706+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:43:34.765706+0000","last_peered":"2023-08-25T04:43:34.765706+0000","last_clean":"2023-08-25T04:43:34.765706+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:43:34.765706+0000","last_undegraded":"2023-08-25T04:43:34.765706+0000","last_fullsized":"2023-08-25T04:43:34.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:25.041276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761550+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:43:34.761550+0000","last_peered":"2023-08-25T04:43:34.761550+0000","last_clean":"2023-08-25T04:43:34.761550+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:43:34.761550+0000","last_undegraded":"2023-08-25T04:43:34.761550+0000","last_fullsized":"2023-08-25T04:43:34.761550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:25.771337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765423+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:43:34.765423+0000","last_peered":"2023-08-25T04:43:34.765423+0000","last_clean":"2023-08-25T04:43:34.765423+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:43:34.765423+0000","last_undegraded":"2023-08-25T04:43:34.765423+0000","last_fullsized":"2023-08-25T04:43:34.765423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:27.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098820+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:43:35.098820+0000","last_peered":"2023-08-25T04:43:35.098820+0000","last_clean":"2023-08-25T04:43:35.098820+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:43:35.098820+0000","last_undegraded":"2023-08-25T04:43:35.098820+0000","last_fullsized":"2023-08-25T04:43:35.098820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:20.999420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761408+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:43:34.761408+0000","last_peered":"2023-08-25T04:43:34.761408+0000","last_clean":"2023-08-25T04:43:34.761408+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:43:34.761408+0000","last_undegraded":"2023-08-25T04:43:34.761408+0000","last_fullsized":"2023-08-25T04:43:34.761408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:14.562518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764854+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:43:34.764854+0000","last_peered":"2023-08-25T04:43:34.764854+0000","last_clean":"2023-08-25T04:43:34.764854+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:43:34.764854+0000","last_undegraded":"2023-08-25T04:43:34.764854+0000","last_fullsized":"2023-08-25T04:43:34.764854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:27.603878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099267+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:43:35.099267+0000","last_peered":"2023-08-25T04:43:35.099267+0000","last_clean":"2023-08-25T04:43:35.099267+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:43:35.099267+0000","last_undegraded":"2023-08-25T04:43:35.099267+0000","last_fullsized":"2023-08-25T04:43:35.099267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:28.457166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769245+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:43:34.769245+0000","last_peered":"2023-08-25T04:43:34.769245+0000","last_clean":"2023-08-25T04:43:34.769245+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:43:34.769245+0000","last_undegraded":"2023-08-25T04:43:34.769245+0000","last_fullsized":"2023-08-25T04:43:34.769245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:47.099556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:56.623150+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:43:56.623150+0000","last_peered":"2023-08-25T04:43:56.623150+0000","last_clean":"2023-08-25T04:43:56.623150+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:43:56.623150+0000","last_undegraded":"2023-08-25T04:43:56.623150+0000","last_fullsized":"2023-08-25T04:43:56.623150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:02.879296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765116+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:43:34.765116+0000","last_peered":"2023-08-25T04:43:34.765116+0000","last_clean":"2023-08-25T04:43:34.765116+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:43:34.765116+0000","last_undegraded":"2023-08-25T04:43:34.765116+0000","last_fullsized":"2023-08-25T04:43:34.765116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:18.764324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769440+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:43:34.769440+0000","last_peered":"2023-08-25T04:43:34.769440+0000","last_clean":"2023-08-25T04:43:34.769440+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:43:34.769440+0000","last_undegraded":"2023-08-25T04:43:34.769440+0000","last_fullsized":"2023-08-25T04:43:34.769440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:38.395114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764497+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:43:34.764497+0000","last_peered":"2023-08-25T04:43:34.764497+0000","last_clean":"2023-08-25T04:43:34.764497+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:43:34.764497+0000","last_undegraded":"2023-08-25T04:43:34.764497+0000","last_fullsized":"2023-08-25T04:43:34.764497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:29:44.193254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099118+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:43:35.099118+0000","last_peered":"2023-08-25T04:43:35.099118+0000","last_clean":"2023-08-25T04:43:35.099118+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:43:35.099118+0000","last_undegraded":"2023-08-25T04:43:35.099118+0000","last_fullsized":"2023-08-25T04:43:35.099118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.013041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765157+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:43:34.765157+0000","last_peered":"2023-08-25T04:43:34.765157+0000","last_clean":"2023-08-25T04:43:34.765157+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:43:34.765157+0000","last_undegraded":"2023-08-25T04:43:34.765157+0000","last_fullsized":"2023-08-25T04:43:34.765157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:14.536121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098445+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:43:35.098445+0000","last_peered":"2023-08-25T04:43:35.098445+0000","last_clean":"2023-08-25T04:43:35.098445+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:43:35.098445+0000","last_undegraded":"2023-08-25T04:43:35.098445+0000","last_fullsized":"2023-08-25T04:43:35.098445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:45.583226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764897+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:43:34.764897+0000","last_peered":"2023-08-25T04:43:34.764897+0000","last_clean":"2023-08-25T04:43:34.764897+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:43:34.764897+0000","last_undegraded":"2023-08-25T04:43:34.764897+0000","last_fullsized":"2023-08-25T04:43:34.764897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:04.925252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764614+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:43:34.764614+0000","last_peered":"2023-08-25T04:43:34.764614+0000","last_clean":"2023-08-25T04:43:34.764614+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:43:34.764614+0000","last_undegraded":"2023-08-25T04:43:34.764614+0000","last_fullsized":"2023-08-25T04:43:34.764614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:24.431309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098697+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:43:35.098697+0000","last_peered":"2023-08-25T04:43:35.098697+0000","last_clean":"2023-08-25T04:43:35.098697+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:43:35.098697+0000","last_undegraded":"2023-08-25T04:43:35.098697+0000","last_fullsized":"2023-08-25T04:43:35.098697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:26.668766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761101+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:43:34.761101+0000","last_peered":"2023-08-25T04:43:34.761101+0000","last_clean":"2023-08-25T04:43:34.761101+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:43:34.761101+0000","last_undegraded":"2023-08-25T04:43:34.761101+0000","last_fullsized":"2023-08-25T04:43:34.761101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:51.398185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765545+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:43:34.765545+0000","last_peered":"2023-08-25T04:43:34.765545+0000","last_clean":"2023-08-25T04:43:34.765545+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:43:34.765545+0000","last_undegraded":"2023-08-25T04:43:34.765545+0000","last_fullsized":"2023-08-25T04:43:34.765545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:28.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:57.624059+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:43:57.624059+0000","last_peered":"2023-08-25T04:43:57.624059+0000","last_clean":"2023-08-25T04:43:57.624059+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:43:57.624059+0000","last_undegraded":"2023-08-25T04:43:57.624059+0000","last_fullsized":"2023-08-25T04:43:57.624059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:44.974722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099154+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:43:35.099154+0000","last_peered":"2023-08-25T04:43:35.099154+0000","last_clean":"2023-08-25T04:43:35.099154+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:43:35.099154+0000","last_undegraded":"2023-08-25T04:43:35.099154+0000","last_fullsized":"2023-08-25T04:43:35.099154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:07.872117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761225+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:43:34.761225+0000","last_peered":"2023-08-25T04:43:34.761225+0000","last_clean":"2023-08-25T04:43:34.761225+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:43:34.761225+0000","last_undegraded":"2023-08-25T04:43:34.761225+0000","last_fullsized":"2023-08-25T04:43:34.761225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:18.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923185+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:43:34.923185+0000","last_peered":"2023-08-25T04:43:34.923185+0000","last_clean":"2023-08-25T04:43:34.923185+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:43:34.923185+0000","last_undegraded":"2023-08-25T04:43:34.923185+0000","last_fullsized":"2023-08-25T04:43:34.923185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:14.764732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766754+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:43:34.766754+0000","last_peered":"2023-08-25T04:43:34.766754+0000","last_clean":"2023-08-25T04:43:34.766754+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:43:34.766754+0000","last_undegraded":"2023-08-25T04:43:34.766754+0000","last_fullsized":"2023-08-25T04:43:34.766754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:01.393459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100106+0000","last_change":"2023-08-25T04:43:35.100106+0000","last_active":"2023-08-25T04:43:35.100106+0000","last_peered":"2023-08-25T04:43:35.100106+0000","last_clean":"2023-08-25T04:43:35.100106+0000","last_became_active":"2023-08-25T04:43:35.099546+0000","last_became_peered":"2023-08-25T04:43:35.099546+0000","last_unstale":"2023-08-25T04:43:35.100106+0000","last_undegraded":"2023-08-25T04:43:35.100106+0000","last_fullsized":"2023-08-25T04:43:35.100106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:25.055771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099268+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:43:35.099268+0000","last_peered":"2023-08-25T04:43:35.099268+0000","last_clean":"2023-08-25T04:43:35.099268+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:43:35.099268+0000","last_undegraded":"2023-08-25T04:43:35.099268+0000","last_fullsized":"2023-08-25T04:43:35.099268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:54.567632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098918+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:43:35.098918+0000","last_peered":"2023-08-25T04:43:35.098918+0000","last_clean":"2023-08-25T04:43:35.098918+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:43:35.098918+0000","last_undegraded":"2023-08-25T04:43:35.098918+0000","last_fullsized":"2023-08-25T04:43:35.098918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:47.455399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761107+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:43:34.761107+0000","last_peered":"2023-08-25T04:43:34.761107+0000","last_clean":"2023-08-25T04:43:34.761107+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:43:34.761107+0000","last_undegraded":"2023-08-25T04:43:34.761107+0000","last_fullsized":"2023-08-25T04:43:34.761107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:27:18.610077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":126,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:58.625467+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:43:58.625467+0000","last_peered":"2023-08-25T04:43:58.625467+0000","last_clean":"2023-08-25T04:43:58.625467+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:43:58.625467+0000","last_undegraded":"2023-08-25T04:43:58.625467+0000","last_fullsized":"2023-08-25T04:43:58.625467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:00.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765074+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:43:34.765074+0000","last_peered":"2023-08-25T04:43:34.765074+0000","last_clean":"2023-08-25T04:43:34.765074+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:43:34.765074+0000","last_undegraded":"2023-08-25T04:43:34.765074+0000","last_fullsized":"2023-08-25T04:43:34.765074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:07.231855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099639+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:43:35.099639+0000","last_peered":"2023-08-25T04:43:35.099639+0000","last_clean":"2023-08-25T04:43:35.099639+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:43:35.099639+0000","last_undegraded":"2023-08-25T04:43:35.099639+0000","last_fullsized":"2023-08-25T04:43:35.099639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:46:34.120949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769705+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:43:34.769705+0000","last_peered":"2023-08-25T04:43:34.769705+0000","last_clean":"2023-08-25T04:43:34.769705+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:43:34.769705+0000","last_undegraded":"2023-08-25T04:43:34.769705+0000","last_fullsized":"2023-08-25T04:43:34.769705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:00.803708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098704+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:43:35.098704+0000","last_peered":"2023-08-25T04:43:35.098704+0000","last_clean":"2023-08-25T04:43:35.098704+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:43:35.098704+0000","last_undegraded":"2023-08-25T04:43:35.098704+0000","last_fullsized":"2023-08-25T04:43:35.098704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:19:57.215688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.767465+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:43:34.767465+0000","last_peered":"2023-08-25T04:43:34.767465+0000","last_clean":"2023-08-25T04:43:34.767465+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:43:34.767465+0000","last_undegraded":"2023-08-25T04:43:34.767465+0000","last_fullsized":"2023-08-25T04:43:34.767465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:22.379000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098949+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:43:35.098949+0000","last_peered":"2023-08-25T04:43:35.098949+0000","last_clean":"2023-08-25T04:43:35.098949+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:43:35.098949+0000","last_undegraded":"2023-08-25T04:43:35.098949+0000","last_fullsized":"2023-08-25T04:43:35.098949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:38.602930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923061+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:43:34.923061+0000","last_peered":"2023-08-25T04:43:34.923061+0000","last_clean":"2023-08-25T04:43:34.923061+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:43:34.923061+0000","last_undegraded":"2023-08-25T04:43:34.923061+0000","last_fullsized":"2023-08-25T04:43:34.923061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:57.719232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761321+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:43:34.761321+0000","last_peered":"2023-08-25T04:43:34.761321+0000","last_clean":"2023-08-25T04:43:34.761321+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:43:34.761321+0000","last_undegraded":"2023-08-25T04:43:34.761321+0000","last_fullsized":"2023-08-25T04:43:34.761321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:45.122188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:56.627203+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:43:56.627203+0000","last_peered":"2023-08-25T04:43:56.627203+0000","last_clean":"2023-08-25T04:43:56.627203+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:43:56.627203+0000","last_undegraded":"2023-08-25T04:43:56.627203+0000","last_fullsized":"2023-08-25T04:43:56.627203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:03:39.958808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922765+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:43:34.922765+0000","last_peered":"2023-08-25T04:43:34.922765+0000","last_clean":"2023-08-25T04:43:34.922765+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:43:34.922765+0000","last_undegraded":"2023-08-25T04:43:34.922765+0000","last_fullsized":"2023-08-25T04:43:34.922765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:22.482680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099199+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:43:35.099199+0000","last_peered":"2023-08-25T04:43:35.099199+0000","last_clean":"2023-08-25T04:43:35.099199+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:43:35.099199+0000","last_undegraded":"2023-08-25T04:43:35.099199+0000","last_fullsized":"2023-08-25T04:43:35.099199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:46.918540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765654+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:43:34.765654+0000","last_peered":"2023-08-25T04:43:34.765654+0000","last_clean":"2023-08-25T04:43:34.765654+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:43:34.765654+0000","last_undegraded":"2023-08-25T04:43:34.765654+0000","last_fullsized":"2023-08-25T04:43:34.765654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:28.897000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:02.027948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098592+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:43:35.098592+0000","last_peered":"2023-08-25T04:43:35.098592+0000","last_clean":"2023-08-25T04:43:35.098592+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:43:35.098592+0000","last_undegraded":"2023-08-25T04:43:35.098592+0000","last_fullsized":"2023-08-25T04:43:35.098592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:19:33.883508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764534+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:43:34.764534+0000","last_peered":"2023-08-25T04:43:34.764534+0000","last_clean":"2023-08-25T04:43:34.764534+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:43:34.764534+0000","last_undegraded":"2023-08-25T04:43:34.764534+0000","last_fullsized":"2023-08-25T04:43:34.764534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:04:26.603958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766834+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:43:34.766834+0000","last_peered":"2023-08-25T04:43:34.766834+0000","last_clean":"2023-08-25T04:43:34.766834+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:43:34.766834+0000","last_undegraded":"2023-08-25T04:43:34.766834+0000","last_fullsized":"2023-08-25T04:43:34.766834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:35:18.420398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764942+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:43:34.764942+0000","last_peered":"2023-08-25T04:43:34.764942+0000","last_clean":"2023-08-25T04:43:34.764942+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:43:34.764942+0000","last_undegraded":"2023-08-25T04:43:34.764942+0000","last_fullsized":"2023-08-25T04:43:34.764942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:31.579981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761767+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:43:34.761767+0000","last_peered":"2023-08-25T04:43:34.761767+0000","last_clean":"2023-08-25T04:43:34.761767+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:43:34.761767+0000","last_undegraded":"2023-08-25T04:43:34.761767+0000","last_fullsized":"2023-08-25T04:43:34.761767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:32.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764777+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:43:34.764777+0000","last_peered":"2023-08-25T04:43:34.764777+0000","last_clean":"2023-08-25T04:43:34.764777+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:43:34.764777+0000","last_undegraded":"2023-08-25T04:43:34.764777+0000","last_fullsized":"2023-08-25T04:43:34.764777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:25:29.024128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099012+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:43:35.099012+0000","last_peered":"2023-08-25T04:43:35.099012+0000","last_clean":"2023-08-25T04:43:35.099012+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:43:35.099012+0000","last_undegraded":"2023-08-25T04:43:35.099012+0000","last_fullsized":"2023-08-25T04:43:35.099012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:17.119675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922819+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:43:34.922819+0000","last_peered":"2023-08-25T04:43:34.922819+0000","last_clean":"2023-08-25T04:43:34.922819+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:43:34.922819+0000","last_undegraded":"2023-08-25T04:43:34.922819+0000","last_fullsized":"2023-08-25T04:43:34.922819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:42.375965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769924+0000","last_change":"2023-08-25T04:43:34.769924+0000","last_active":"2023-08-25T04:43:34.769924+0000","last_peered":"2023-08-25T04:43:34.769924+0000","last_clean":"2023-08-25T04:43:34.769924+0000","last_became_active":"2023-08-25T04:43:34.769575+0000","last_became_peered":"2023-08-25T04:43:34.769575+0000","last_unstale":"2023-08-25T04:43:34.769924+0000","last_undegraded":"2023-08-25T04:43:34.769924+0000","last_fullsized":"2023-08-25T04:43:34.769924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:32.013614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099091+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:43:35.099091+0000","last_peered":"2023-08-25T04:43:35.099091+0000","last_clean":"2023-08-25T04:43:35.099091+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:43:35.099091+0000","last_undegraded":"2023-08-25T04:43:35.099091+0000","last_fullsized":"2023-08-25T04:43:35.099091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:32:02.438964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.772835+0000","last_change":"2023-08-25T04:43:34.771233+0000","last_active":"2023-08-25T04:43:34.772835+0000","last_peered":"2023-08-25T04:43:34.772835+0000","last_clean":"2023-08-25T04:43:34.772835+0000","last_became_active":"2023-08-25T04:43:34.770887+0000","last_became_peered":"2023-08-25T04:43:34.770887+0000","last_unstale":"2023-08-25T04:43:34.772835+0000","last_undegraded":"2023-08-25T04:43:34.772835+0000","last_fullsized":"2023-08-25T04:43:34.772835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:08.119078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:57.627945+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:43:57.627945+0000","last_peered":"2023-08-25T04:43:57.627945+0000","last_clean":"2023-08-25T04:43:57.627945+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:43:57.627945+0000","last_undegraded":"2023-08-25T04:43:57.627945+0000","last_fullsized":"2023-08-25T04:43:57.627945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:43.271628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922975+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:43:34.922975+0000","last_peered":"2023-08-25T04:43:34.922975+0000","last_clean":"2023-08-25T04:43:34.922975+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:43:34.922975+0000","last_undegraded":"2023-08-25T04:43:34.922975+0000","last_fullsized":"2023-08-25T04:43:34.922975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:04.252263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764688+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:43:34.764688+0000","last_peered":"2023-08-25T04:43:34.764688+0000","last_clean":"2023-08-25T04:43:34.764688+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:43:34.764688+0000","last_undegraded":"2023-08-25T04:43:34.764688+0000","last_fullsized":"2023-08-25T04:43:34.764688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:33:20.016878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761743+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:43:34.761743+0000","last_peered":"2023-08-25T04:43:34.761743+0000","last_clean":"2023-08-25T04:43:34.761743+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:43:34.761743+0000","last_undegraded":"2023-08-25T04:43:34.761743+0000","last_fullsized":"2023-08-25T04:43:34.761743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:10.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922583+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:43:34.922583+0000","last_peered":"2023-08-25T04:43:34.922583+0000","last_clean":"2023-08-25T04:43:34.922583+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:43:34.922583+0000","last_undegraded":"2023-08-25T04:43:34.922583+0000","last_fullsized":"2023-08-25T04:43:34.922583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:20.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099295+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:43:35.099295+0000","last_peered":"2023-08-25T04:43:35.099295+0000","last_clean":"2023-08-25T04:43:35.099295+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:43:35.099295+0000","last_undegraded":"2023-08-25T04:43:35.099295+0000","last_fullsized":"2023-08-25T04:43:35.099295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.332881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098261+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:43:35.098261+0000","last_peered":"2023-08-25T04:43:35.098261+0000","last_clean":"2023-08-25T04:43:35.098261+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:43:35.098261+0000","last_undegraded":"2023-08-25T04:43:35.098261+0000","last_fullsized":"2023-08-25T04:43:35.098261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:51:31.902430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922724+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:43:34.922724+0000","last_peered":"2023-08-25T04:43:34.922724+0000","last_clean":"2023-08-25T04:43:34.922724+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:43:34.922724+0000","last_undegraded":"2023-08-25T04:43:34.922724+0000","last_fullsized":"2023-08-25T04:43:34.922724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:47.704937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765952+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:43:34.765952+0000","last_peered":"2023-08-25T04:43:34.765952+0000","last_clean":"2023-08-25T04:43:34.765952+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:43:34.765952+0000","last_undegraded":"2023-08-25T04:43:34.765952+0000","last_fullsized":"2023-08-25T04:43:34.765952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:38:36.745026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765218+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:43:34.765218+0000","last_peered":"2023-08-25T04:43:34.765218+0000","last_clean":"2023-08-25T04:43:34.765218+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:43:34.765218+0000","last_undegraded":"2023-08-25T04:43:34.765218+0000","last_fullsized":"2023-08-25T04:43:34.765218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:15:24.598228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:14:26.761777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764760+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:43:34.764760+0000","last_peered":"2023-08-25T04:43:34.764760+0000","last_clean":"2023-08-25T04:43:34.764760+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:43:34.764760+0000","last_undegraded":"2023-08-25T04:43:34.764760+0000","last_fullsized":"2023-08-25T04:43:34.764760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:23.436529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.777294+0000","last_change":"2023-08-25T04:43:34.777294+0000","last_active":"2023-08-25T04:43:34.777294+0000","last_peered":"2023-08-25T04:43:34.777294+0000","last_clean":"2023-08-25T04:43:34.777294+0000","last_became_active":"2023-08-25T04:43:34.768036+0000","last_became_peered":"2023-08-25T04:43:34.768036+0000","last_unstale":"2023-08-25T04:43:34.777294+0000","last_undegraded":"2023-08-25T04:43:34.777294+0000","last_fullsized":"2023-08-25T04:43:34.777294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:52.521174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765705+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:43:34.765705+0000","last_peered":"2023-08-25T04:43:34.765705+0000","last_clean":"2023-08-25T04:43:34.765705+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:43:34.765705+0000","last_undegraded":"2023-08-25T04:43:34.765705+0000","last_fullsized":"2023-08-25T04:43:34.765705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:34.577696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768990+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:43:34.768990+0000","last_peered":"2023-08-25T04:43:34.768990+0000","last_clean":"2023-08-25T04:43:34.768990+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:43:34.768990+0000","last_undegraded":"2023-08-25T04:43:34.768990+0000","last_fullsized":"2023-08-25T04:43:34.768990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:09.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761921+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:43:34.761921+0000","last_peered":"2023-08-25T04:43:34.761921+0000","last_clean":"2023-08-25T04:43:34.761921+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:43:34.761921+0000","last_undegraded":"2023-08-25T04:43:34.761921+0000","last_fullsized":"2023-08-25T04:43:34.761921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:16.228950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764889+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:43:34.764889+0000","last_peered":"2023-08-25T04:43:34.764889+0000","last_clean":"2023-08-25T04:43:34.764889+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:43:34.764889+0000","last_undegraded":"2023-08-25T04:43:34.764889+0000","last_fullsized":"2023-08-25T04:43:34.764889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:14.043346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768669+0000","last_change":"2023-08-25T04:43:34.768669+0000","last_active":"2023-08-25T04:43:34.768669+0000","last_peered":"2023-08-25T04:43:34.768669+0000","last_clean":"2023-08-25T04:43:34.768669+0000","last_became_active":"2023-08-25T04:43:34.768275+0000","last_became_peered":"2023-08-25T04:43:34.768275+0000","last_unstale":"2023-08-25T04:43:34.768669+0000","last_undegraded":"2023-08-25T04:43:34.768669+0000","last_fullsized":"2023-08-25T04:43:34.768669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:39:53.898292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,11],"acting":[4,8,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098506+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:43:35.098506+0000","last_peered":"2023-08-25T04:43:35.098506+0000","last_clean":"2023-08-25T04:43:35.098506+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:43:35.098506+0000","last_undegraded":"2023-08-25T04:43:35.098506+0000","last_fullsized":"2023-08-25T04:43:35.098506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:19.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764296+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:43:34.764296+0000","last_peered":"2023-08-25T04:43:34.764296+0000","last_clean":"2023-08-25T04:43:34.764296+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:43:34.764296+0000","last_undegraded":"2023-08-25T04:43:34.764296+0000","last_fullsized":"2023-08-25T04:43:34.764296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:11:04.726743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761549+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:43:34.761549+0000","last_peered":"2023-08-25T04:43:34.761549+0000","last_clean":"2023-08-25T04:43:34.761549+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:43:34.761549+0000","last_undegraded":"2023-08-25T04:43:34.761549+0000","last_fullsized":"2023-08-25T04:43:34.761549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:48.355591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765742+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:43:34.765742+0000","last_peered":"2023-08-25T04:43:34.765742+0000","last_clean":"2023-08-25T04:43:34.765742+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:43:34.765742+0000","last_undegraded":"2023-08-25T04:43:34.765742+0000","last_fullsized":"2023-08-25T04:43:34.765742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:04.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766384+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:43:34.766384+0000","last_peered":"2023-08-25T04:43:34.766384+0000","last_clean":"2023-08-25T04:43:34.766384+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:43:34.766384+0000","last_undegraded":"2023-08-25T04:43:34.766384+0000","last_fullsized":"2023-08-25T04:43:34.766384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:02:40.882696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761618+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:43:34.761618+0000","last_peered":"2023-08-25T04:43:34.761618+0000","last_clean":"2023-08-25T04:43:34.761618+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:43:34.761618+0000","last_undegraded":"2023-08-25T04:43:34.761618+0000","last_fullsized":"2023-08-25T04:43:34.761618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:23.594133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.779656+0000","last_change":"2023-08-25T04:43:34.779656+0000","last_active":"2023-08-25T04:43:34.779656+0000","last_peered":"2023-08-25T04:43:34.779656+0000","last_clean":"2023-08-25T04:43:34.779656+0000","last_became_active":"2023-08-25T04:43:34.770821+0000","last_became_peered":"2023-08-25T04:43:34.770821+0000","last_unstale":"2023-08-25T04:43:34.779656+0000","last_undegraded":"2023-08-25T04:43:34.779656+0000","last_fullsized":"2023-08-25T04:43:34.779656+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:08:31.402768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766129+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:43:34.766129+0000","last_peered":"2023-08-25T04:43:34.766129+0000","last_clean":"2023-08-25T04:43:34.766129+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:43:34.766129+0000","last_undegraded":"2023-08-25T04:43:34.766129+0000","last_fullsized":"2023-08-25T04:43:34.766129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:18.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761763+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:43:34.761763+0000","last_peered":"2023-08-25T04:43:34.761763+0000","last_clean":"2023-08-25T04:43:34.761763+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:43:34.761763+0000","last_undegraded":"2023-08-25T04:43:34.761763+0000","last_fullsized":"2023-08-25T04:43:34.761763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:14.914419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765951+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:43:34.765951+0000","last_peered":"2023-08-25T04:43:34.765951+0000","last_clean":"2023-08-25T04:43:34.765951+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:43:34.765951+0000","last_undegraded":"2023-08-25T04:43:34.765951+0000","last_fullsized":"2023-08-25T04:43:34.765951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:07.289831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922875+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:43:34.922875+0000","last_peered":"2023-08-25T04:43:34.922875+0000","last_clean":"2023-08-25T04:43:34.922875+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:43:34.922875+0000","last_undegraded":"2023-08-25T04:43:34.922875+0000","last_fullsized":"2023-08-25T04:43:34.922875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:13.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098664+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:43:35.098664+0000","last_peered":"2023-08-25T04:43:35.098664+0000","last_clean":"2023-08-25T04:43:35.098664+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:43:35.098664+0000","last_undegraded":"2023-08-25T04:43:35.098664+0000","last_fullsized":"2023-08-25T04:43:35.098664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:02:41.742505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769588+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:43:34.769588+0000","last_peered":"2023-08-25T04:43:34.769588+0000","last_clean":"2023-08-25T04:43:34.769588+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:43:34.769588+0000","last_undegraded":"2023-08-25T04:43:34.769588+0000","last_fullsized":"2023-08-25T04:43:34.769588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:56.254087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923325+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:43:34.923325+0000","last_peered":"2023-08-25T04:43:34.923325+0000","last_clean":"2023-08-25T04:43:34.923325+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:43:34.923325+0000","last_undegraded":"2023-08-25T04:43:34.923325+0000","last_fullsized":"2023-08-25T04:43:34.923325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:27.560915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764860+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:43:34.764860+0000","last_peered":"2023-08-25T04:43:34.764860+0000","last_clean":"2023-08-25T04:43:34.764860+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:43:34.764860+0000","last_undegraded":"2023-08-25T04:43:34.764860+0000","last_fullsized":"2023-08-25T04:43:34.764860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:05.061144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099452+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:43:35.099452+0000","last_peered":"2023-08-25T04:43:35.099452+0000","last_clean":"2023-08-25T04:43:35.099452+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:43:35.099452+0000","last_undegraded":"2023-08-25T04:43:35.099452+0000","last_fullsized":"2023-08-25T04:43:35.099452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:28.319336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923060+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:43:34.923060+0000","last_peered":"2023-08-25T04:43:34.923060+0000","last_clean":"2023-08-25T04:43:34.923060+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:43:34.923060+0000","last_undegraded":"2023-08-25T04:43:34.923060+0000","last_fullsized":"2023-08-25T04:43:34.923060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:25.511917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098448+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:43:35.098448+0000","last_peered":"2023-08-25T04:43:35.098448+0000","last_clean":"2023-08-25T04:43:35.098448+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:43:35.098448+0000","last_undegraded":"2023-08-25T04:43:35.098448+0000","last_fullsized":"2023-08-25T04:43:35.098448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:15.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099187+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:43:35.099187+0000","last_peered":"2023-08-25T04:43:35.099187+0000","last_clean":"2023-08-25T04:43:35.099187+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:43:35.099187+0000","last_undegraded":"2023-08-25T04:43:35.099187+0000","last_fullsized":"2023-08-25T04:43:35.099187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:00.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923329+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:43:34.923329+0000","last_peered":"2023-08-25T04:43:34.923329+0000","last_clean":"2023-08-25T04:43:34.923329+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:43:34.923329+0000","last_undegraded":"2023-08-25T04:43:34.923329+0000","last_fullsized":"2023-08-25T04:43:34.923329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:03.933897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766391+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:43:34.766391+0000","last_peered":"2023-08-25T04:43:34.766391+0000","last_clean":"2023-08-25T04:43:34.766391+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:43:34.766391+0000","last_undegraded":"2023-08-25T04:43:34.766391+0000","last_fullsized":"2023-08-25T04:43:34.766391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:42.835852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769164+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:43:34.769164+0000","last_peered":"2023-08-25T04:43:34.769164+0000","last_clean":"2023-08-25T04:43:34.769164+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:43:34.769164+0000","last_undegraded":"2023-08-25T04:43:34.769164+0000","last_fullsized":"2023-08-25T04:43:34.769164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:38.408069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769152+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:43:34.769152+0000","last_peered":"2023-08-25T04:43:34.769152+0000","last_clean":"2023-08-25T04:43:34.769152+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:43:34.769152+0000","last_undegraded":"2023-08-25T04:43:34.769152+0000","last_fullsized":"2023-08-25T04:43:34.769152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:25.658427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100100+0000","last_change":"2023-08-25T04:43:35.100100+0000","last_active":"2023-08-25T04:43:35.100100+0000","last_peered":"2023-08-25T04:43:35.100100+0000","last_clean":"2023-08-25T04:43:35.100100+0000","last_became_active":"2023-08-25T04:43:35.099734+0000","last_became_peered":"2023-08-25T04:43:35.099734+0000","last_unstale":"2023-08-25T04:43:35.100100+0000","last_undegraded":"2023-08-25T04:43:35.100100+0000","last_fullsized":"2023-08-25T04:43:35.100100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:17.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099291+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:43:35.099291+0000","last_peered":"2023-08-25T04:43:35.099291+0000","last_clean":"2023-08-25T04:43:35.099291+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:43:35.099291+0000","last_undegraded":"2023-08-25T04:43:35.099291+0000","last_fullsized":"2023-08-25T04:43:35.099291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:05:39.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766561+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:43:34.766561+0000","last_peered":"2023-08-25T04:43:34.766561+0000","last_clean":"2023-08-25T04:43:34.766561+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:43:34.766561+0000","last_undegraded":"2023-08-25T04:43:34.766561+0000","last_fullsized":"2023-08-25T04:43:34.766561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:47:14.892189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770202+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:43:34.770202+0000","last_peered":"2023-08-25T04:43:34.770202+0000","last_clean":"2023-08-25T04:43:34.770202+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:43:34.770202+0000","last_undegraded":"2023-08-25T04:43:34.770202+0000","last_fullsized":"2023-08-25T04:43:34.770202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:31.428150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765901+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:43:34.765901+0000","last_peered":"2023-08-25T04:43:34.765901+0000","last_clean":"2023-08-25T04:43:34.765901+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:43:34.765901+0000","last_undegraded":"2023-08-25T04:43:34.765901+0000","last_fullsized":"2023-08-25T04:43:34.765901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:04.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:13.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765879+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:43:34.765879+0000","last_peered":"2023-08-25T04:43:34.765879+0000","last_clean":"2023-08-25T04:43:34.765879+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:43:34.765879+0000","last_undegraded":"2023-08-25T04:43:34.765879+0000","last_fullsized":"2023-08-25T04:43:34.765879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:03:35.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:15:06.086740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098646+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:43:35.098646+0000","last_peered":"2023-08-25T04:43:35.098646+0000","last_clean":"2023-08-25T04:43:35.098646+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:43:35.098646+0000","last_undegraded":"2023-08-25T04:43:35.098646+0000","last_fullsized":"2023-08-25T04:43:35.098646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:42.910594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.771119+0000","last_change":"2023-08-25T04:43:34.771119+0000","last_active":"2023-08-25T04:43:34.771119+0000","last_peered":"2023-08-25T04:43:34.771119+0000","last_clean":"2023-08-25T04:43:34.771119+0000","last_became_active":"2023-08-25T04:43:34.770717+0000","last_became_peered":"2023-08-25T04:43:34.770717+0000","last_unstale":"2023-08-25T04:43:34.771119+0000","last_undegraded":"2023-08-25T04:43:34.771119+0000","last_fullsized":"2023-08-25T04:43:34.771119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:42.596435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":276,"num_read_kb":152,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":483313,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.873}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92800000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96599999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678046,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1078519,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.503}]},{"osd":1,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.625}]},{"osd":5,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91100000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.036,"15min":1.036},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.91000000000000003,"5min":1.167,"15min":1.167},"last":0.81399999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83999999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841570,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483549,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.59999999999999998,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.083,"15min":1.083},"last":0.38},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67100000000000004,"5min":0.873,"15min":0.873},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":1.089,"15min":1.089},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78400000000000003,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.85999999999999999,"5min":1.266,"15min":1.266},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.628},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93500000000000005,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.70999999999999996,"15min":0.73699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.84899999999999998,"15min":0.879},"min":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90000000000000002,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.81000000000000005,"15min":0.82799999999999996},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93600000000000005,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.84699999999999998,"15min":0.86699999999999999},"min":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.92200000000000004,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84599999999999997,"15min":0.86599999999999999},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96599999999999997,"5min":1.371,"15min":1.371},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.92000000000000004,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.91200000000000003,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.91000000000000003,"15min":0.93600000000000005},"min":{"1min":0.59699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":1.488,"15min":1.488},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.60099999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97499999999999998,"5min":1.413,"15min":1.413},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.94299999999999995,"15min":0.97099999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.026,"5min":1.603,"15min":1.603},"last":0.66200000000000003}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.47,"15min":1.47},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68100000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.70899999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103207,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483138,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.378},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.29599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77800000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.623}]},{"osd":9,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.77400000000000002,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.61899999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69299999999999995}]},{"osd":10,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.751,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.755,"15min":0.74399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65700000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074560,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.41099999999999998},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.371},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39700000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.876,"15min":0.876},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.628},"min":{"1min":0.51900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71799999999999997,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":0.996,"15min":0.996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93200000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.53700000000000003}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96899999999999997,"5min":1.109,"15min":1.109},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93500000000000005,"5min":1.014,"15min":1.014},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.78200000000000003,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96599999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69499999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83999999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482821,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51500000000000001,"15min":0.48799999999999999},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.433}]},{"osd":1,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.65500000000000003},"min":{"1min":0.55000000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.28,"15min":1.28},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73099999999999998,"15min":0.70599999999999996},"min":{"1min":0.59999999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96199999999999997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.623}]},{"osd":5,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89900000000000002,"5min":1.208,"15min":1.208},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.502}]},{"osd":7,"last update":"Fri Aug 25 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.76500000000000001,"15min":0.73199999999999998},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.56499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.753}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.177,"5min":1.389,"15min":1.389},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.877,"15min":0.88900000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.89300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.66700000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91300000000000003,"15min":0.90600000000000003},"min":{"1min":0.68700000000000006,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.77000000000000002}]},{"osd":11,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.70899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":486300,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.54700000000000004},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84299999999999997,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54800000000000004,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.54200000000000004,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.153,"5min":1.548,"15min":1.548},"last":0.62}]},{"osd":3,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.432,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Aug 25 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.46700000000000003,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.48099999999999998,"15min":0.42899999999999999},"min":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.35799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":1.613,"15min":1.613},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89700000000000002,"5min":1.597,"15min":1.597},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.437}]},{"osd":7,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.073,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.49099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.38500000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70399999999999996}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.628,"5min":0.437,"15min":0.437},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.78500000000000003,"15min":0.76700000000000002},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78700000000000003,"15min":0.76900000000000002},"min":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.66600000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.76100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":484209,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.58399999999999996,"15min":0.51700000000000002},"min":{"1min":0.41699999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.56899999999999995},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":1.194,"15min":1.194},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.877}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.747}]},{"osd":4,"last update":"Fri Aug 25 04:43:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.441,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84999999999999998,"5min":1.488,"15min":1.488},"last":1.167},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94299999999999995,"5min":1.603,"15min":1.603},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99199999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97499999999999998,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.044},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.121}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.056,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0840000000000001,"5min":1.365,"15min":1.365},"last":0.80500000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0489999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.004}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82699999999999996,"15min":0.81699999999999995},"min":{"1min":0.54700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.1399999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.74299999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0569999999999999,"5min":1.214,"15min":1.214},"last":1.0169999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.92400000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":487923,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.755},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.53000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.436,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.58299999999999996},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Aug 25 04:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.59999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.04},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.014},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.97999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483015,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.39200000000000002},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58199999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.42099999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.64700000000000002,"5min":0.745,"15min":0.745},"last":0.501}]},{"osd":1,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.879},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.58699999999999997},"min":{"1min":0.56100000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.60099999999999998},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89300000000000002,"5min":1.53,"15min":1.53},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":1.573,"15min":1.573},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65700000000000003,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80900000000000005,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85699999999999998,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85299999999999998,"5min":1.556,"15min":1.556},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.53100000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.84299999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.747,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.337,"15min":1.337},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.65300000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.274,"15min":1.274},"last":0.93200000000000005}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.88900000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1020000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.83899999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":483375,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.35499999999999998},"min":{"1min":0.34599999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67400000000000004,"5min":0.753,"15min":0.753},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.36899999999999999},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501}]},{"osd":2,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.99,"15min":1.99},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88800000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.874,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.755}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93400000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.874}]},{"osd":9,"last update":"Fri Aug 25 04:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92500000000000004,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.97999999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1074575,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.53600000000000003,"15min":0.5},"min":{"1min":0.432,"5min":0.318,"15min":0.318},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.30299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.39400000000000002},"min":{"1min":0.30499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Aug 25 04:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80300000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.52600000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.60799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.495}]},{"osd":9,"last update":"Fri Aug 25 04:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97199999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64400000000000002}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.61599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.72999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:44:00.405 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph pg dump --format=json 2023-08-25T04:44:00.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:00 smithi145 ceph-mon[97681]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:00.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:00 smithi145 ceph-mon[97681]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:00.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:00 smithi162 ceph-mon[130941]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:00.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:00 smithi162 ceph-mon[130941]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:00.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:00 smithi183 ceph-mon[131414]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:00.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:00 smithi183 ceph-mon[131414]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:01.749 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:01 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.749 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:01 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.750 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:01 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:44:01.750 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:01 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:44:01.750 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:01 smithi145 ceph-mon[97681]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:01 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:01 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:01 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:44:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:01 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:44:01.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:01 smithi162 ceph-mon[130941]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:01 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:01 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:01.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:01 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:44:01.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:01 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:44:01.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:01 smithi183 ceph-mon[131414]: from='mgr.14150 172.21.15.145:0/414322988' entity='mgr.x' 2023-08-25T04:44:02.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 -- 172.21.15.145:0/232814573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8106940 msgr2=0x7fa8d8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:02.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/232814573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8106940 0x7fa8d8106d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009990 tx=0x7fa8c8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 -- 172.21.15.145:0/232814573 shutdown_connections 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/232814573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8101300 0x7fa8d810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/232814573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8106940 0x7fa8d8106d20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/232814573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 0x7fa8d8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 -- 172.21.15.145:0/232814573 >> 172.21.15.145:0/232814573 conn(0x7fa8d80fc410 msgr2=0x7fa8d80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 -- 172.21.15.145:0/232814573 shutdown_connections 2023-08-25T04:44:02.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.634+0000 7fa8de4dd700 1 -- 172.21.15.145:0/232814573 wait complete. 2023-08-25T04:44:02.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.635+0000 7fa8de4dd700 1 Processor -- start 2023-08-25T04:44:02.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.635+0000 7fa8de4dd700 1 -- start start 2023-08-25T04:44:02.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 0x7fa8d8072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:02.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:02.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 0x7fa8d80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:02.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a4f10 con 0x7fa8d8101300 2023-08-25T04:44:02.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a5080 con 0x7fa8d8106940 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8de4dd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a5240 con 0x7fa8d8100940 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8d7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8d7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51662/0 (socket says 172.21.15.145:51662) 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8d7fff700 1 -- 172.21.15.145:0/3624434352 learned_addr learned my addr 172.21.15.145:0/3624434352 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8dca7a700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 0x7fa8d80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.636+0000 7fa8cffff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 0x7fa8d8072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:02.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 -- 172.21.15.145:0/3624434352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 msgr2=0x7fa8d8072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 0x7fa8d8072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 -- 172.21.15.145:0/3624434352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 msgr2=0x7fa8d80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 0x7fa8d80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 -- 172.21.15.145:0/3624434352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c80095f0 con 0x7fa8d8101300 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8dca7a700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 0x7fa8d80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d7fff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d980 tx=0x7fa8c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004220 con 0x7fa8d8101300 2023-08-25T04:44:02.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d81a5520 con 0x7fa8d8101300 2023-08-25T04:44:02.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c0007640 con 0x7fa8d8101300 2023-08-25T04:44:02.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0005690 con 0x7fa8d8101300 2023-08-25T04:44:02.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.637+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d81a5a70 con 0x7fa8d8101300 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.638+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d804e620 con 0x7fa8d8101300 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.640+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa8c0004750 con 0x7fa8d8101300 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.642+0000 7fa8d5ffb700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 0x7fa8c40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.642+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa8c00cb0a0 con 0x7fa8d8101300 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.642+0000 7fa8cffff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 0x7fa8c40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:02.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.642+0000 7fa8cffff700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 0x7fa8c40a2410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800b5c0 tx=0x7fa8c8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:02.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.646+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa8c0094b30 con 0x7fa8d8101300 2023-08-25T04:44:02.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.841+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8d8110be0 con 0x7fa8c409ff50 2023-08-25T04:44:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.853+0000 7fa8d5ffb700 1 -- 172.21.15.145:0/3624434352 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426978 (secure 0 0 0) 0x7fa8d8110be0 con 0x7fa8c409ff50 2023-08-25T04:44:02.862 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:02.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 msgr2=0x7fa8c40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 0x7fa8c40a2410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800b5c0 tx=0x7fa8c8005fd0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 msgr2=0x7fa8d8073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d980 tx=0x7fa8c000dc90 comp rx=0 tx=0).stop 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 shutdown_connections 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa8c409ff50 0x7fa8c40a2410 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d8106940 0x7fa8d80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8d8100940 0x7fa8d8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 --2- 172.21.15.145:0/3624434352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8d8101300 0x7fa8d8073160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:02.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.859+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 >> 172.21.15.145:0/3624434352 conn(0x7fa8d80fc410 msgr2=0x7fa8d80fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:02.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.860+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 shutdown_connections 2023-08-25T04:44:02.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:02.860+0000 7fa8de4dd700 1 -- 172.21.15.145:0/3624434352 wait complete. 2023-08-25T04:44:02.869 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:44:02.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:02 smithi162 ceph-mon[130941]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:02.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:02 smithi183 ceph-mon[131414]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:03.042 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:02 smithi145 ceph-mon[97681]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:03.332 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-08-25T04:44:01.701928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":412,"num_read_kb":271,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335924,"kb_used_data":11652,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605900,"statfs":{"total":1151940427776,"available":1151596441600,"internally_reserved":0,"allocated":11931648,"data_stored":7585297,"data_compressed":318375,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003310"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765465+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:43:34.765465+0000","last_peered":"2023-08-25T04:43:34.765465+0000","last_clean":"2023-08-25T04:43:34.765465+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:43:34.765465+0000","last_undegraded":"2023-08-25T04:43:34.765465+0000","last_fullsized":"2023-08-25T04:43:34.765465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:59.835651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764768+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:43:34.764768+0000","last_peered":"2023-08-25T04:43:34.764768+0000","last_clean":"2023-08-25T04:43:34.764768+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:43:34.764768+0000","last_undegraded":"2023-08-25T04:43:34.764768+0000","last_fullsized":"2023-08-25T04:43:34.764768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:56:09.848052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766305+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:43:34.766305+0000","last_peered":"2023-08-25T04:43:34.766305+0000","last_clean":"2023-08-25T04:43:34.766305+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:43:34.766305+0000","last_undegraded":"2023-08-25T04:43:34.766305+0000","last_fullsized":"2023-08-25T04:43:34.766305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:56.683325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762343+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:43:34.762343+0000","last_peered":"2023-08-25T04:43:34.762343+0000","last_clean":"2023-08-25T04:43:34.762343+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:43:34.762343+0000","last_undegraded":"2023-08-25T04:43:34.762343+0000","last_fullsized":"2023-08-25T04:43:34.762343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:57:35.343381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766096+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:43:34.766096+0000","last_peered":"2023-08-25T04:43:34.766096+0000","last_clean":"2023-08-25T04:43:34.766096+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:43:34.766096+0000","last_undegraded":"2023-08-25T04:43:34.766096+0000","last_fullsized":"2023-08-25T04:43:34.766096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:53:26.027535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764535+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:43:34.764535+0000","last_peered":"2023-08-25T04:43:34.764535+0000","last_clean":"2023-08-25T04:43:34.764535+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:43:34.764535+0000","last_undegraded":"2023-08-25T04:43:34.764535+0000","last_fullsized":"2023-08-25T04:43:34.764535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:00.744928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922718+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:43:34.922718+0000","last_peered":"2023-08-25T04:43:34.922718+0000","last_clean":"2023-08-25T04:43:34.922718+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:43:34.922718+0000","last_undegraded":"2023-08-25T04:43:34.922718+0000","last_fullsized":"2023-08-25T04:43:34.922718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:47:46.522466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765008+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:43:34.765008+0000","last_peered":"2023-08-25T04:43:34.765008+0000","last_clean":"2023-08-25T04:43:34.765008+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:43:34.765008+0000","last_undegraded":"2023-08-25T04:43:34.765008+0000","last_fullsized":"2023-08-25T04:43:34.765008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:08:38.855500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099337+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:43:35.099337+0000","last_peered":"2023-08-25T04:43:35.099337+0000","last_clean":"2023-08-25T04:43:35.099337+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:43:35.099337+0000","last_undegraded":"2023-08-25T04:43:35.099337+0000","last_fullsized":"2023-08-25T04:43:35.099337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:36.964244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765117+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:43:34.765117+0000","last_peered":"2023-08-25T04:43:34.765117+0000","last_clean":"2023-08-25T04:43:34.765117+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:43:34.765117+0000","last_undegraded":"2023-08-25T04:43:34.765117+0000","last_fullsized":"2023-08-25T04:43:34.765117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:00.051214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923189+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:43:34.923189+0000","last_peered":"2023-08-25T04:43:34.923189+0000","last_clean":"2023-08-25T04:43:34.923189+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:43:34.923189+0000","last_undegraded":"2023-08-25T04:43:34.923189+0000","last_fullsized":"2023-08-25T04:43:34.923189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:24.087589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762199+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:43:34.762199+0000","last_peered":"2023-08-25T04:43:34.762199+0000","last_clean":"2023-08-25T04:43:34.762199+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:43:34.762199+0000","last_undegraded":"2023-08-25T04:43:34.762199+0000","last_fullsized":"2023-08-25T04:43:34.762199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:23.029577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099138+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:43:35.099138+0000","last_peered":"2023-08-25T04:43:35.099138+0000","last_clean":"2023-08-25T04:43:35.099138+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:43:35.099138+0000","last_undegraded":"2023-08-25T04:43:35.099138+0000","last_fullsized":"2023-08-25T04:43:35.099138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:53.794589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764677+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:43:34.764677+0000","last_peered":"2023-08-25T04:43:34.764677+0000","last_clean":"2023-08-25T04:43:34.764677+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:43:34.764677+0000","last_undegraded":"2023-08-25T04:43:34.764677+0000","last_fullsized":"2023-08-25T04:43:34.764677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:07.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764876+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:43:34.764876+0000","last_peered":"2023-08-25T04:43:34.764876+0000","last_clean":"2023-08-25T04:43:34.764876+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:43:34.764876+0000","last_undegraded":"2023-08-25T04:43:34.764876+0000","last_fullsized":"2023-08-25T04:43:34.764876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:15:49.317438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098418+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:43:35.098418+0000","last_peered":"2023-08-25T04:43:35.098418+0000","last_clean":"2023-08-25T04:43:35.098418+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:43:35.098418+0000","last_undegraded":"2023-08-25T04:43:35.098418+0000","last_fullsized":"2023-08-25T04:43:35.098418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:53.277068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765316+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:43:34.765316+0000","last_peered":"2023-08-25T04:43:34.765316+0000","last_clean":"2023-08-25T04:43:34.765316+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:43:34.765316+0000","last_undegraded":"2023-08-25T04:43:34.765316+0000","last_fullsized":"2023-08-25T04:43:34.765316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:04.508809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761810+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:43:34.761810+0000","last_peered":"2023-08-25T04:43:34.761810+0000","last_clean":"2023-08-25T04:43:34.761810+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:43:34.761810+0000","last_undegraded":"2023-08-25T04:43:34.761810+0000","last_fullsized":"2023-08-25T04:43:34.761810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:10:02.382799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770010+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:43:34.770010+0000","last_peered":"2023-08-25T04:43:34.770010+0000","last_clean":"2023-08-25T04:43:34.770010+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:43:34.770010+0000","last_undegraded":"2023-08-25T04:43:34.770010+0000","last_fullsized":"2023-08-25T04:43:34.770010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:00:34.306505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769760+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:43:34.769760+0000","last_peered":"2023-08-25T04:43:34.769760+0000","last_clean":"2023-08-25T04:43:34.769760+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:43:34.769760+0000","last_undegraded":"2023-08-25T04:43:34.769760+0000","last_fullsized":"2023-08-25T04:43:34.769760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:23.911843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761620+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:43:34.761620+0000","last_peered":"2023-08-25T04:43:34.761620+0000","last_clean":"2023-08-25T04:43:34.761620+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:43:34.761620+0000","last_undegraded":"2023-08-25T04:43:34.761620+0000","last_fullsized":"2023-08-25T04:43:34.761620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:30.639568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765547+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:43:34.765547+0000","last_peered":"2023-08-25T04:43:34.765547+0000","last_clean":"2023-08-25T04:43:34.765547+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:43:34.765547+0000","last_undegraded":"2023-08-25T04:43:34.765547+0000","last_fullsized":"2023-08-25T04:43:34.765547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:24.659622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100513+0000","last_change":"2023-08-25T04:43:35.100513+0000","last_active":"2023-08-25T04:43:35.100513+0000","last_peered":"2023-08-25T04:43:35.100513+0000","last_clean":"2023-08-25T04:43:35.100513+0000","last_became_active":"2023-08-25T04:43:35.100067+0000","last_became_peered":"2023-08-25T04:43:35.100067+0000","last_unstale":"2023-08-25T04:43:35.100513+0000","last_undegraded":"2023-08-25T04:43:35.100513+0000","last_fullsized":"2023-08-25T04:43:35.100513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:23:03.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765693+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:43:34.765693+0000","last_peered":"2023-08-25T04:43:34.765693+0000","last_clean":"2023-08-25T04:43:34.765693+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:43:34.765693+0000","last_undegraded":"2023-08-25T04:43:34.765693+0000","last_fullsized":"2023-08-25T04:43:34.765693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:38:39.362078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765706+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:43:34.765706+0000","last_peered":"2023-08-25T04:43:34.765706+0000","last_clean":"2023-08-25T04:43:34.765706+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:43:34.765706+0000","last_undegraded":"2023-08-25T04:43:34.765706+0000","last_fullsized":"2023-08-25T04:43:34.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:25.041276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761550+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:43:34.761550+0000","last_peered":"2023-08-25T04:43:34.761550+0000","last_clean":"2023-08-25T04:43:34.761550+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:43:34.761550+0000","last_undegraded":"2023-08-25T04:43:34.761550+0000","last_fullsized":"2023-08-25T04:43:34.761550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:25.771337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765423+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:43:34.765423+0000","last_peered":"2023-08-25T04:43:34.765423+0000","last_clean":"2023-08-25T04:43:34.765423+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:43:34.765423+0000","last_undegraded":"2023-08-25T04:43:34.765423+0000","last_fullsized":"2023-08-25T04:43:34.765423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:27.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098820+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:43:35.098820+0000","last_peered":"2023-08-25T04:43:35.098820+0000","last_clean":"2023-08-25T04:43:35.098820+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:43:35.098820+0000","last_undegraded":"2023-08-25T04:43:35.098820+0000","last_fullsized":"2023-08-25T04:43:35.098820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:20.999420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761408+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:43:34.761408+0000","last_peered":"2023-08-25T04:43:34.761408+0000","last_clean":"2023-08-25T04:43:34.761408+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:43:34.761408+0000","last_undegraded":"2023-08-25T04:43:34.761408+0000","last_fullsized":"2023-08-25T04:43:34.761408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:14.562518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764854+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:43:34.764854+0000","last_peered":"2023-08-25T04:43:34.764854+0000","last_clean":"2023-08-25T04:43:34.764854+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:43:34.764854+0000","last_undegraded":"2023-08-25T04:43:34.764854+0000","last_fullsized":"2023-08-25T04:43:34.764854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:27.603878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099267+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:43:35.099267+0000","last_peered":"2023-08-25T04:43:35.099267+0000","last_clean":"2023-08-25T04:43:35.099267+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:43:35.099267+0000","last_undegraded":"2023-08-25T04:43:35.099267+0000","last_fullsized":"2023-08-25T04:43:35.099267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:28.457166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769245+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:43:34.769245+0000","last_peered":"2023-08-25T04:43:34.769245+0000","last_clean":"2023-08-25T04:43:34.769245+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:43:34.769245+0000","last_undegraded":"2023-08-25T04:43:34.769245+0000","last_fullsized":"2023-08-25T04:43:34.769245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:47.099556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:56.623150+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:43:56.623150+0000","last_peered":"2023-08-25T04:43:56.623150+0000","last_clean":"2023-08-25T04:43:56.623150+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:43:56.623150+0000","last_undegraded":"2023-08-25T04:43:56.623150+0000","last_fullsized":"2023-08-25T04:43:56.623150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:02.879296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765116+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:43:34.765116+0000","last_peered":"2023-08-25T04:43:34.765116+0000","last_clean":"2023-08-25T04:43:34.765116+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:43:34.765116+0000","last_undegraded":"2023-08-25T04:43:34.765116+0000","last_fullsized":"2023-08-25T04:43:34.765116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:18.764324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769440+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:43:34.769440+0000","last_peered":"2023-08-25T04:43:34.769440+0000","last_clean":"2023-08-25T04:43:34.769440+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:43:34.769440+0000","last_undegraded":"2023-08-25T04:43:34.769440+0000","last_fullsized":"2023-08-25T04:43:34.769440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:38.395114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764497+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:43:34.764497+0000","last_peered":"2023-08-25T04:43:34.764497+0000","last_clean":"2023-08-25T04:43:34.764497+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:43:34.764497+0000","last_undegraded":"2023-08-25T04:43:34.764497+0000","last_fullsized":"2023-08-25T04:43:34.764497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:29:44.193254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099118+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:43:35.099118+0000","last_peered":"2023-08-25T04:43:35.099118+0000","last_clean":"2023-08-25T04:43:35.099118+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:43:35.099118+0000","last_undegraded":"2023-08-25T04:43:35.099118+0000","last_fullsized":"2023-08-25T04:43:35.099118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.013041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765157+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:43:34.765157+0000","last_peered":"2023-08-25T04:43:34.765157+0000","last_clean":"2023-08-25T04:43:34.765157+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:43:34.765157+0000","last_undegraded":"2023-08-25T04:43:34.765157+0000","last_fullsized":"2023-08-25T04:43:34.765157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:14.536121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098445+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:43:35.098445+0000","last_peered":"2023-08-25T04:43:35.098445+0000","last_clean":"2023-08-25T04:43:35.098445+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:43:35.098445+0000","last_undegraded":"2023-08-25T04:43:35.098445+0000","last_fullsized":"2023-08-25T04:43:35.098445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:45.583226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764897+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:43:34.764897+0000","last_peered":"2023-08-25T04:43:34.764897+0000","last_clean":"2023-08-25T04:43:34.764897+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:43:34.764897+0000","last_undegraded":"2023-08-25T04:43:34.764897+0000","last_fullsized":"2023-08-25T04:43:34.764897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:04.925252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764614+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:43:34.764614+0000","last_peered":"2023-08-25T04:43:34.764614+0000","last_clean":"2023-08-25T04:43:34.764614+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:43:34.764614+0000","last_undegraded":"2023-08-25T04:43:34.764614+0000","last_fullsized":"2023-08-25T04:43:34.764614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:24.431309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098697+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:43:35.098697+0000","last_peered":"2023-08-25T04:43:35.098697+0000","last_clean":"2023-08-25T04:43:35.098697+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:43:35.098697+0000","last_undegraded":"2023-08-25T04:43:35.098697+0000","last_fullsized":"2023-08-25T04:43:35.098697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:26.668766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761101+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:43:34.761101+0000","last_peered":"2023-08-25T04:43:34.761101+0000","last_clean":"2023-08-25T04:43:34.761101+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:43:34.761101+0000","last_undegraded":"2023-08-25T04:43:34.761101+0000","last_fullsized":"2023-08-25T04:43:34.761101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:51.398185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765545+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:43:34.765545+0000","last_peered":"2023-08-25T04:43:34.765545+0000","last_clean":"2023-08-25T04:43:34.765545+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:43:34.765545+0000","last_undegraded":"2023-08-25T04:43:34.765545+0000","last_fullsized":"2023-08-25T04:43:34.765545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:28.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:57.624059+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:43:57.624059+0000","last_peered":"2023-08-25T04:43:57.624059+0000","last_clean":"2023-08-25T04:43:57.624059+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:43:57.624059+0000","last_undegraded":"2023-08-25T04:43:57.624059+0000","last_fullsized":"2023-08-25T04:43:57.624059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:44.974722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099154+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:43:35.099154+0000","last_peered":"2023-08-25T04:43:35.099154+0000","last_clean":"2023-08-25T04:43:35.099154+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:43:35.099154+0000","last_undegraded":"2023-08-25T04:43:35.099154+0000","last_fullsized":"2023-08-25T04:43:35.099154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:07.872117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761225+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:43:34.761225+0000","last_peered":"2023-08-25T04:43:34.761225+0000","last_clean":"2023-08-25T04:43:34.761225+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:43:34.761225+0000","last_undegraded":"2023-08-25T04:43:34.761225+0000","last_fullsized":"2023-08-25T04:43:34.761225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:18.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923185+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:43:34.923185+0000","last_peered":"2023-08-25T04:43:34.923185+0000","last_clean":"2023-08-25T04:43:34.923185+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:43:34.923185+0000","last_undegraded":"2023-08-25T04:43:34.923185+0000","last_fullsized":"2023-08-25T04:43:34.923185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:14.764732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766754+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:43:34.766754+0000","last_peered":"2023-08-25T04:43:34.766754+0000","last_clean":"2023-08-25T04:43:34.766754+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:43:34.766754+0000","last_undegraded":"2023-08-25T04:43:34.766754+0000","last_fullsized":"2023-08-25T04:43:34.766754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:01.393459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100106+0000","last_change":"2023-08-25T04:43:35.100106+0000","last_active":"2023-08-25T04:43:35.100106+0000","last_peered":"2023-08-25T04:43:35.100106+0000","last_clean":"2023-08-25T04:43:35.100106+0000","last_became_active":"2023-08-25T04:43:35.099546+0000","last_became_peered":"2023-08-25T04:43:35.099546+0000","last_unstale":"2023-08-25T04:43:35.100106+0000","last_undegraded":"2023-08-25T04:43:35.100106+0000","last_fullsized":"2023-08-25T04:43:35.100106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:25.055771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099268+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:43:35.099268+0000","last_peered":"2023-08-25T04:43:35.099268+0000","last_clean":"2023-08-25T04:43:35.099268+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:43:35.099268+0000","last_undegraded":"2023-08-25T04:43:35.099268+0000","last_fullsized":"2023-08-25T04:43:35.099268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:54.567632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098918+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:43:35.098918+0000","last_peered":"2023-08-25T04:43:35.098918+0000","last_clean":"2023-08-25T04:43:35.098918+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:43:35.098918+0000","last_undegraded":"2023-08-25T04:43:35.098918+0000","last_fullsized":"2023-08-25T04:43:35.098918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:47.455399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761107+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:43:34.761107+0000","last_peered":"2023-08-25T04:43:34.761107+0000","last_clean":"2023-08-25T04:43:34.761107+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:43:34.761107+0000","last_undegraded":"2023-08-25T04:43:34.761107+0000","last_fullsized":"2023-08-25T04:43:34.761107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:27:18.610077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":126,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:58.625467+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:43:58.625467+0000","last_peered":"2023-08-25T04:43:58.625467+0000","last_clean":"2023-08-25T04:43:58.625467+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:43:58.625467+0000","last_undegraded":"2023-08-25T04:43:58.625467+0000","last_fullsized":"2023-08-25T04:43:58.625467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:00.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765074+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:43:34.765074+0000","last_peered":"2023-08-25T04:43:34.765074+0000","last_clean":"2023-08-25T04:43:34.765074+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:43:34.765074+0000","last_undegraded":"2023-08-25T04:43:34.765074+0000","last_fullsized":"2023-08-25T04:43:34.765074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:07.231855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099639+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:43:35.099639+0000","last_peered":"2023-08-25T04:43:35.099639+0000","last_clean":"2023-08-25T04:43:35.099639+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:43:35.099639+0000","last_undegraded":"2023-08-25T04:43:35.099639+0000","last_fullsized":"2023-08-25T04:43:35.099639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:46:34.120949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769705+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:43:34.769705+0000","last_peered":"2023-08-25T04:43:34.769705+0000","last_clean":"2023-08-25T04:43:34.769705+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:43:34.769705+0000","last_undegraded":"2023-08-25T04:43:34.769705+0000","last_fullsized":"2023-08-25T04:43:34.769705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:00.803708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098704+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:43:35.098704+0000","last_peered":"2023-08-25T04:43:35.098704+0000","last_clean":"2023-08-25T04:43:35.098704+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:43:35.098704+0000","last_undegraded":"2023-08-25T04:43:35.098704+0000","last_fullsized":"2023-08-25T04:43:35.098704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:19:57.215688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.767465+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:43:34.767465+0000","last_peered":"2023-08-25T04:43:34.767465+0000","last_clean":"2023-08-25T04:43:34.767465+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:43:34.767465+0000","last_undegraded":"2023-08-25T04:43:34.767465+0000","last_fullsized":"2023-08-25T04:43:34.767465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:22.379000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098949+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:43:35.098949+0000","last_peered":"2023-08-25T04:43:35.098949+0000","last_clean":"2023-08-25T04:43:35.098949+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:43:35.098949+0000","last_undegraded":"2023-08-25T04:43:35.098949+0000","last_fullsized":"2023-08-25T04:43:35.098949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:38.602930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923061+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:43:34.923061+0000","last_peered":"2023-08-25T04:43:34.923061+0000","last_clean":"2023-08-25T04:43:34.923061+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:43:34.923061+0000","last_undegraded":"2023-08-25T04:43:34.923061+0000","last_fullsized":"2023-08-25T04:43:34.923061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:57.719232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761321+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:43:34.761321+0000","last_peered":"2023-08-25T04:43:34.761321+0000","last_clean":"2023-08-25T04:43:34.761321+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:43:34.761321+0000","last_undegraded":"2023-08-25T04:43:34.761321+0000","last_fullsized":"2023-08-25T04:43:34.761321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:45.122188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:56.627203+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:43:56.627203+0000","last_peered":"2023-08-25T04:43:56.627203+0000","last_clean":"2023-08-25T04:43:56.627203+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:43:56.627203+0000","last_undegraded":"2023-08-25T04:43:56.627203+0000","last_fullsized":"2023-08-25T04:43:56.627203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:03:39.958808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922765+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:43:34.922765+0000","last_peered":"2023-08-25T04:43:34.922765+0000","last_clean":"2023-08-25T04:43:34.922765+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:43:34.922765+0000","last_undegraded":"2023-08-25T04:43:34.922765+0000","last_fullsized":"2023-08-25T04:43:34.922765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:22.482680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099199+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:43:35.099199+0000","last_peered":"2023-08-25T04:43:35.099199+0000","last_clean":"2023-08-25T04:43:35.099199+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:43:35.099199+0000","last_undegraded":"2023-08-25T04:43:35.099199+0000","last_fullsized":"2023-08-25T04:43:35.099199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:46.918540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765654+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:43:34.765654+0000","last_peered":"2023-08-25T04:43:34.765654+0000","last_clean":"2023-08-25T04:43:34.765654+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:43:34.765654+0000","last_undegraded":"2023-08-25T04:43:34.765654+0000","last_fullsized":"2023-08-25T04:43:34.765654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:28.897000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:02.027948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098592+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:43:35.098592+0000","last_peered":"2023-08-25T04:43:35.098592+0000","last_clean":"2023-08-25T04:43:35.098592+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:43:35.098592+0000","last_undegraded":"2023-08-25T04:43:35.098592+0000","last_fullsized":"2023-08-25T04:43:35.098592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:19:33.883508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764534+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:43:34.764534+0000","last_peered":"2023-08-25T04:43:34.764534+0000","last_clean":"2023-08-25T04:43:34.764534+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:43:34.764534+0000","last_undegraded":"2023-08-25T04:43:34.764534+0000","last_fullsized":"2023-08-25T04:43:34.764534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:04:26.603958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766834+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:43:34.766834+0000","last_peered":"2023-08-25T04:43:34.766834+0000","last_clean":"2023-08-25T04:43:34.766834+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:43:34.766834+0000","last_undegraded":"2023-08-25T04:43:34.766834+0000","last_fullsized":"2023-08-25T04:43:34.766834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:35:18.420398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764942+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:43:34.764942+0000","last_peered":"2023-08-25T04:43:34.764942+0000","last_clean":"2023-08-25T04:43:34.764942+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:43:34.764942+0000","last_undegraded":"2023-08-25T04:43:34.764942+0000","last_fullsized":"2023-08-25T04:43:34.764942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:31.579981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761767+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:43:34.761767+0000","last_peered":"2023-08-25T04:43:34.761767+0000","last_clean":"2023-08-25T04:43:34.761767+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:43:34.761767+0000","last_undegraded":"2023-08-25T04:43:34.761767+0000","last_fullsized":"2023-08-25T04:43:34.761767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:32.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764777+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:43:34.764777+0000","last_peered":"2023-08-25T04:43:34.764777+0000","last_clean":"2023-08-25T04:43:34.764777+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:43:34.764777+0000","last_undegraded":"2023-08-25T04:43:34.764777+0000","last_fullsized":"2023-08-25T04:43:34.764777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:25:29.024128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099012+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:43:35.099012+0000","last_peered":"2023-08-25T04:43:35.099012+0000","last_clean":"2023-08-25T04:43:35.099012+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:43:35.099012+0000","last_undegraded":"2023-08-25T04:43:35.099012+0000","last_fullsized":"2023-08-25T04:43:35.099012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:17.119675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922819+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:43:34.922819+0000","last_peered":"2023-08-25T04:43:34.922819+0000","last_clean":"2023-08-25T04:43:34.922819+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:43:34.922819+0000","last_undegraded":"2023-08-25T04:43:34.922819+0000","last_fullsized":"2023-08-25T04:43:34.922819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:42.375965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769924+0000","last_change":"2023-08-25T04:43:34.769924+0000","last_active":"2023-08-25T04:43:34.769924+0000","last_peered":"2023-08-25T04:43:34.769924+0000","last_clean":"2023-08-25T04:43:34.769924+0000","last_became_active":"2023-08-25T04:43:34.769575+0000","last_became_peered":"2023-08-25T04:43:34.769575+0000","last_unstale":"2023-08-25T04:43:34.769924+0000","last_undegraded":"2023-08-25T04:43:34.769924+0000","last_fullsized":"2023-08-25T04:43:34.769924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:32.013614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099091+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:43:35.099091+0000","last_peered":"2023-08-25T04:43:35.099091+0000","last_clean":"2023-08-25T04:43:35.099091+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:43:35.099091+0000","last_undegraded":"2023-08-25T04:43:35.099091+0000","last_fullsized":"2023-08-25T04:43:35.099091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:32:02.438964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.772835+0000","last_change":"2023-08-25T04:43:34.771233+0000","last_active":"2023-08-25T04:43:34.772835+0000","last_peered":"2023-08-25T04:43:34.772835+0000","last_clean":"2023-08-25T04:43:34.772835+0000","last_became_active":"2023-08-25T04:43:34.770887+0000","last_became_peered":"2023-08-25T04:43:34.770887+0000","last_unstale":"2023-08-25T04:43:34.772835+0000","last_undegraded":"2023-08-25T04:43:34.772835+0000","last_fullsized":"2023-08-25T04:43:34.772835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:08.119078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:57.627945+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:43:57.627945+0000","last_peered":"2023-08-25T04:43:57.627945+0000","last_clean":"2023-08-25T04:43:57.627945+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:43:57.627945+0000","last_undegraded":"2023-08-25T04:43:57.627945+0000","last_fullsized":"2023-08-25T04:43:57.627945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:43.271628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922975+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:43:34.922975+0000","last_peered":"2023-08-25T04:43:34.922975+0000","last_clean":"2023-08-25T04:43:34.922975+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:43:34.922975+0000","last_undegraded":"2023-08-25T04:43:34.922975+0000","last_fullsized":"2023-08-25T04:43:34.922975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:04.252263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764688+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:43:34.764688+0000","last_peered":"2023-08-25T04:43:34.764688+0000","last_clean":"2023-08-25T04:43:34.764688+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:43:34.764688+0000","last_undegraded":"2023-08-25T04:43:34.764688+0000","last_fullsized":"2023-08-25T04:43:34.764688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:33:20.016878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761743+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:43:34.761743+0000","last_peered":"2023-08-25T04:43:34.761743+0000","last_clean":"2023-08-25T04:43:34.761743+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:43:34.761743+0000","last_undegraded":"2023-08-25T04:43:34.761743+0000","last_fullsized":"2023-08-25T04:43:34.761743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:10.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922583+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:43:34.922583+0000","last_peered":"2023-08-25T04:43:34.922583+0000","last_clean":"2023-08-25T04:43:34.922583+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:43:34.922583+0000","last_undegraded":"2023-08-25T04:43:34.922583+0000","last_fullsized":"2023-08-25T04:43:34.922583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:20.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099295+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:43:35.099295+0000","last_peered":"2023-08-25T04:43:35.099295+0000","last_clean":"2023-08-25T04:43:35.099295+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:43:35.099295+0000","last_undegraded":"2023-08-25T04:43:35.099295+0000","last_fullsized":"2023-08-25T04:43:35.099295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.332881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098261+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:43:35.098261+0000","last_peered":"2023-08-25T04:43:35.098261+0000","last_clean":"2023-08-25T04:43:35.098261+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:43:35.098261+0000","last_undegraded":"2023-08-25T04:43:35.098261+0000","last_fullsized":"2023-08-25T04:43:35.098261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:51:31.902430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922724+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:43:34.922724+0000","last_peered":"2023-08-25T04:43:34.922724+0000","last_clean":"2023-08-25T04:43:34.922724+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:43:34.922724+0000","last_undegraded":"2023-08-25T04:43:34.922724+0000","last_fullsized":"2023-08-25T04:43:34.922724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:47.704937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765952+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:43:34.765952+0000","last_peered":"2023-08-25T04:43:34.765952+0000","last_clean":"2023-08-25T04:43:34.765952+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:43:34.765952+0000","last_undegraded":"2023-08-25T04:43:34.765952+0000","last_fullsized":"2023-08-25T04:43:34.765952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:38:36.745026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765218+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:43:34.765218+0000","last_peered":"2023-08-25T04:43:34.765218+0000","last_clean":"2023-08-25T04:43:34.765218+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:43:34.765218+0000","last_undegraded":"2023-08-25T04:43:34.765218+0000","last_fullsized":"2023-08-25T04:43:34.765218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:15:24.598228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:14:26.761777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764760+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:43:34.764760+0000","last_peered":"2023-08-25T04:43:34.764760+0000","last_clean":"2023-08-25T04:43:34.764760+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:43:34.764760+0000","last_undegraded":"2023-08-25T04:43:34.764760+0000","last_fullsized":"2023-08-25T04:43:34.764760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:23.436529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.777294+0000","last_change":"2023-08-25T04:43:34.777294+0000","last_active":"2023-08-25T04:43:34.777294+0000","last_peered":"2023-08-25T04:43:34.777294+0000","last_clean":"2023-08-25T04:43:34.777294+0000","last_became_active":"2023-08-25T04:43:34.768036+0000","last_became_peered":"2023-08-25T04:43:34.768036+0000","last_unstale":"2023-08-25T04:43:34.777294+0000","last_undegraded":"2023-08-25T04:43:34.777294+0000","last_fullsized":"2023-08-25T04:43:34.777294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:52.521174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765705+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:43:34.765705+0000","last_peered":"2023-08-25T04:43:34.765705+0000","last_clean":"2023-08-25T04:43:34.765705+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:43:34.765705+0000","last_undegraded":"2023-08-25T04:43:34.765705+0000","last_fullsized":"2023-08-25T04:43:34.765705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:34.577696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768990+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:43:34.768990+0000","last_peered":"2023-08-25T04:43:34.768990+0000","last_clean":"2023-08-25T04:43:34.768990+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:43:34.768990+0000","last_undegraded":"2023-08-25T04:43:34.768990+0000","last_fullsized":"2023-08-25T04:43:34.768990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:09.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761921+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:43:34.761921+0000","last_peered":"2023-08-25T04:43:34.761921+0000","last_clean":"2023-08-25T04:43:34.761921+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:43:34.761921+0000","last_undegraded":"2023-08-25T04:43:34.761921+0000","last_fullsized":"2023-08-25T04:43:34.761921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:16.228950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764889+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:43:34.764889+0000","last_peered":"2023-08-25T04:43:34.764889+0000","last_clean":"2023-08-25T04:43:34.764889+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:43:34.764889+0000","last_undegraded":"2023-08-25T04:43:34.764889+0000","last_fullsized":"2023-08-25T04:43:34.764889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:14.043346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768669+0000","last_change":"2023-08-25T04:43:34.768669+0000","last_active":"2023-08-25T04:43:34.768669+0000","last_peered":"2023-08-25T04:43:34.768669+0000","last_clean":"2023-08-25T04:43:34.768669+0000","last_became_active":"2023-08-25T04:43:34.768275+0000","last_became_peered":"2023-08-25T04:43:34.768275+0000","last_unstale":"2023-08-25T04:43:34.768669+0000","last_undegraded":"2023-08-25T04:43:34.768669+0000","last_fullsized":"2023-08-25T04:43:34.768669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:39:53.898292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,11],"acting":[4,8,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098506+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:43:35.098506+0000","last_peered":"2023-08-25T04:43:35.098506+0000","last_clean":"2023-08-25T04:43:35.098506+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:43:35.098506+0000","last_undegraded":"2023-08-25T04:43:35.098506+0000","last_fullsized":"2023-08-25T04:43:35.098506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:19.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764296+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:43:34.764296+0000","last_peered":"2023-08-25T04:43:34.764296+0000","last_clean":"2023-08-25T04:43:34.764296+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:43:34.764296+0000","last_undegraded":"2023-08-25T04:43:34.764296+0000","last_fullsized":"2023-08-25T04:43:34.764296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:11:04.726743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761549+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:43:34.761549+0000","last_peered":"2023-08-25T04:43:34.761549+0000","last_clean":"2023-08-25T04:43:34.761549+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:43:34.761549+0000","last_undegraded":"2023-08-25T04:43:34.761549+0000","last_fullsized":"2023-08-25T04:43:34.761549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:48.355591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765742+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:43:34.765742+0000","last_peered":"2023-08-25T04:43:34.765742+0000","last_clean":"2023-08-25T04:43:34.765742+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:43:34.765742+0000","last_undegraded":"2023-08-25T04:43:34.765742+0000","last_fullsized":"2023-08-25T04:43:34.765742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:04.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766384+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:43:34.766384+0000","last_peered":"2023-08-25T04:43:34.766384+0000","last_clean":"2023-08-25T04:43:34.766384+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:43:34.766384+0000","last_undegraded":"2023-08-25T04:43:34.766384+0000","last_fullsized":"2023-08-25T04:43:34.766384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:02:40.882696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761618+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:43:34.761618+0000","last_peered":"2023-08-25T04:43:34.761618+0000","last_clean":"2023-08-25T04:43:34.761618+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:43:34.761618+0000","last_undegraded":"2023-08-25T04:43:34.761618+0000","last_fullsized":"2023-08-25T04:43:34.761618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:23.594133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.779656+0000","last_change":"2023-08-25T04:43:34.779656+0000","last_active":"2023-08-25T04:43:34.779656+0000","last_peered":"2023-08-25T04:43:34.779656+0000","last_clean":"2023-08-25T04:43:34.779656+0000","last_became_active":"2023-08-25T04:43:34.770821+0000","last_became_peered":"2023-08-25T04:43:34.770821+0000","last_unstale":"2023-08-25T04:43:34.779656+0000","last_undegraded":"2023-08-25T04:43:34.779656+0000","last_fullsized":"2023-08-25T04:43:34.779656+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:08:31.402768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766129+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:43:34.766129+0000","last_peered":"2023-08-25T04:43:34.766129+0000","last_clean":"2023-08-25T04:43:34.766129+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:43:34.766129+0000","last_undegraded":"2023-08-25T04:43:34.766129+0000","last_fullsized":"2023-08-25T04:43:34.766129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:18.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761763+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:43:34.761763+0000","last_peered":"2023-08-25T04:43:34.761763+0000","last_clean":"2023-08-25T04:43:34.761763+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:43:34.761763+0000","last_undegraded":"2023-08-25T04:43:34.761763+0000","last_fullsized":"2023-08-25T04:43:34.761763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:14.914419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765951+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:43:34.765951+0000","last_peered":"2023-08-25T04:43:34.765951+0000","last_clean":"2023-08-25T04:43:34.765951+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:43:34.765951+0000","last_undegraded":"2023-08-25T04:43:34.765951+0000","last_fullsized":"2023-08-25T04:43:34.765951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:07.289831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922875+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:43:34.922875+0000","last_peered":"2023-08-25T04:43:34.922875+0000","last_clean":"2023-08-25T04:43:34.922875+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:43:34.922875+0000","last_undegraded":"2023-08-25T04:43:34.922875+0000","last_fullsized":"2023-08-25T04:43:34.922875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:13.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098664+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:43:35.098664+0000","last_peered":"2023-08-25T04:43:35.098664+0000","last_clean":"2023-08-25T04:43:35.098664+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:43:35.098664+0000","last_undegraded":"2023-08-25T04:43:35.098664+0000","last_fullsized":"2023-08-25T04:43:35.098664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:02:41.742505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769588+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:43:34.769588+0000","last_peered":"2023-08-25T04:43:34.769588+0000","last_clean":"2023-08-25T04:43:34.769588+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:43:34.769588+0000","last_undegraded":"2023-08-25T04:43:34.769588+0000","last_fullsized":"2023-08-25T04:43:34.769588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:56.254087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923325+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:43:34.923325+0000","last_peered":"2023-08-25T04:43:34.923325+0000","last_clean":"2023-08-25T04:43:34.923325+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:43:34.923325+0000","last_undegraded":"2023-08-25T04:43:34.923325+0000","last_fullsized":"2023-08-25T04:43:34.923325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:27.560915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764860+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:43:34.764860+0000","last_peered":"2023-08-25T04:43:34.764860+0000","last_clean":"2023-08-25T04:43:34.764860+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:43:34.764860+0000","last_undegraded":"2023-08-25T04:43:34.764860+0000","last_fullsized":"2023-08-25T04:43:34.764860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:05.061144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099452+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:43:35.099452+0000","last_peered":"2023-08-25T04:43:35.099452+0000","last_clean":"2023-08-25T04:43:35.099452+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:43:35.099452+0000","last_undegraded":"2023-08-25T04:43:35.099452+0000","last_fullsized":"2023-08-25T04:43:35.099452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:28.319336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923060+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:43:34.923060+0000","last_peered":"2023-08-25T04:43:34.923060+0000","last_clean":"2023-08-25T04:43:34.923060+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:43:34.923060+0000","last_undegraded":"2023-08-25T04:43:34.923060+0000","last_fullsized":"2023-08-25T04:43:34.923060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:25.511917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098448+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:43:35.098448+0000","last_peered":"2023-08-25T04:43:35.098448+0000","last_clean":"2023-08-25T04:43:35.098448+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:43:35.098448+0000","last_undegraded":"2023-08-25T04:43:35.098448+0000","last_fullsized":"2023-08-25T04:43:35.098448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:15.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099187+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:43:35.099187+0000","last_peered":"2023-08-25T04:43:35.099187+0000","last_clean":"2023-08-25T04:43:35.099187+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:43:35.099187+0000","last_undegraded":"2023-08-25T04:43:35.099187+0000","last_fullsized":"2023-08-25T04:43:35.099187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:00.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923329+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:43:34.923329+0000","last_peered":"2023-08-25T04:43:34.923329+0000","last_clean":"2023-08-25T04:43:34.923329+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:43:34.923329+0000","last_undegraded":"2023-08-25T04:43:34.923329+0000","last_fullsized":"2023-08-25T04:43:34.923329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:03.933897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766391+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:43:34.766391+0000","last_peered":"2023-08-25T04:43:34.766391+0000","last_clean":"2023-08-25T04:43:34.766391+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:43:34.766391+0000","last_undegraded":"2023-08-25T04:43:34.766391+0000","last_fullsized":"2023-08-25T04:43:34.766391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:42.835852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769164+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:43:34.769164+0000","last_peered":"2023-08-25T04:43:34.769164+0000","last_clean":"2023-08-25T04:43:34.769164+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:43:34.769164+0000","last_undegraded":"2023-08-25T04:43:34.769164+0000","last_fullsized":"2023-08-25T04:43:34.769164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:38.408069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769152+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:43:34.769152+0000","last_peered":"2023-08-25T04:43:34.769152+0000","last_clean":"2023-08-25T04:43:34.769152+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:43:34.769152+0000","last_undegraded":"2023-08-25T04:43:34.769152+0000","last_fullsized":"2023-08-25T04:43:34.769152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:25.658427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100100+0000","last_change":"2023-08-25T04:43:35.100100+0000","last_active":"2023-08-25T04:43:35.100100+0000","last_peered":"2023-08-25T04:43:35.100100+0000","last_clean":"2023-08-25T04:43:35.100100+0000","last_became_active":"2023-08-25T04:43:35.099734+0000","last_became_peered":"2023-08-25T04:43:35.099734+0000","last_unstale":"2023-08-25T04:43:35.100100+0000","last_undegraded":"2023-08-25T04:43:35.100100+0000","last_fullsized":"2023-08-25T04:43:35.100100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:17.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099291+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:43:35.099291+0000","last_peered":"2023-08-25T04:43:35.099291+0000","last_clean":"2023-08-25T04:43:35.099291+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:43:35.099291+0000","last_undegraded":"2023-08-25T04:43:35.099291+0000","last_fullsized":"2023-08-25T04:43:35.099291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:05:39.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766561+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:43:34.766561+0000","last_peered":"2023-08-25T04:43:34.766561+0000","last_clean":"2023-08-25T04:43:34.766561+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:43:34.766561+0000","last_undegraded":"2023-08-25T04:43:34.766561+0000","last_fullsized":"2023-08-25T04:43:34.766561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:47:14.892189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770202+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:43:34.770202+0000","last_peered":"2023-08-25T04:43:34.770202+0000","last_clean":"2023-08-25T04:43:34.770202+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:43:34.770202+0000","last_undegraded":"2023-08-25T04:43:34.770202+0000","last_fullsized":"2023-08-25T04:43:34.770202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:31.428150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765901+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:43:34.765901+0000","last_peered":"2023-08-25T04:43:34.765901+0000","last_clean":"2023-08-25T04:43:34.765901+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:43:34.765901+0000","last_undegraded":"2023-08-25T04:43:34.765901+0000","last_fullsized":"2023-08-25T04:43:34.765901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:04.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:13.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765879+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:43:34.765879+0000","last_peered":"2023-08-25T04:43:34.765879+0000","last_clean":"2023-08-25T04:43:34.765879+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:43:34.765879+0000","last_undegraded":"2023-08-25T04:43:34.765879+0000","last_fullsized":"2023-08-25T04:43:34.765879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:03:35.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:15:06.086740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098646+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:43:35.098646+0000","last_peered":"2023-08-25T04:43:35.098646+0000","last_clean":"2023-08-25T04:43:35.098646+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:43:35.098646+0000","last_undegraded":"2023-08-25T04:43:35.098646+0000","last_fullsized":"2023-08-25T04:43:35.098646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:42.910594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.771119+0000","last_change":"2023-08-25T04:43:34.771119+0000","last_active":"2023-08-25T04:43:34.771119+0000","last_peered":"2023-08-25T04:43:34.771119+0000","last_clean":"2023-08-25T04:43:34.771119+0000","last_became_active":"2023-08-25T04:43:34.770717+0000","last_became_peered":"2023-08-25T04:43:34.770717+0000","last_unstale":"2023-08-25T04:43:34.771119+0000","last_undegraded":"2023-08-25T04:43:34.771119+0000","last_fullsized":"2023-08-25T04:43:34.771119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:42.596435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":276,"num_read_kb":152,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":483313,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.873}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92800000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.97499999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96599999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678046,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1078519,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.503}]},{"osd":1,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.625}]},{"osd":5,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91100000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.036,"15min":1.036},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.91000000000000003,"5min":1.167,"15min":1.167},"last":0.81399999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.83999999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841570,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483549,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.59999999999999998,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.083,"15min":1.083},"last":0.38},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67100000000000004,"5min":0.873,"15min":0.873},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":1.089,"15min":1.089},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78400000000000003,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.85999999999999999,"5min":1.266,"15min":1.266},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.628},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93500000000000005,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.70999999999999996,"15min":0.73699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82999999999999996,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.84899999999999998,"15min":0.879},"min":{"1min":0.54800000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90000000000000002,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.81000000000000005,"15min":0.82799999999999996},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93600000000000005,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.84699999999999998,"15min":0.86699999999999999},"min":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.92200000000000004,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84599999999999997,"15min":0.86599999999999999},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96599999999999997,"5min":1.371,"15min":1.371},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.92000000000000004,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.91200000000000003,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.91000000000000003,"15min":0.93600000000000005},"min":{"1min":0.59699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":1.488,"15min":1.488},"last":0.67100000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.60099999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97499999999999998,"5min":1.413,"15min":1.413},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.94299999999999995,"15min":0.97099999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.026,"5min":1.603,"15min":1.603},"last":0.66200000000000003}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.47,"15min":1.47},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68100000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.70899999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103207,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483138,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.45500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.378},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.29599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77800000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.51400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.623}]},{"osd":9,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.77400000000000002,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.61899999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69299999999999995}]},{"osd":10,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.751,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.755,"15min":0.74399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65700000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074560,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.41099999999999998},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.371},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39700000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.876,"15min":0.876},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.628},"min":{"1min":0.51900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71799999999999997,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":0.996,"15min":0.996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93200000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.53700000000000003}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96899999999999997,"5min":1.109,"15min":1.109},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93500000000000005,"5min":1.014,"15min":1.014},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.78200000000000003,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96599999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69499999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83999999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482821,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51500000000000001,"15min":0.48799999999999999},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.433}]},{"osd":1,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.65500000000000003},"min":{"1min":0.55000000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.28,"15min":1.28},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73099999999999998,"15min":0.70599999999999996},"min":{"1min":0.59999999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96199999999999997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.623}]},{"osd":5,"last update":"Fri Aug 25 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89900000000000002,"5min":1.208,"15min":1.208},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.502}]},{"osd":7,"last update":"Fri Aug 25 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.76500000000000001,"15min":0.73199999999999998},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81299999999999994,"15min":0.78700000000000003},"min":{"1min":0.56499999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.753}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.177,"5min":1.389,"15min":1.389},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.877,"15min":0.88900000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88900000000000001,"15min":0.89300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.66700000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91300000000000003,"15min":0.90600000000000003},"min":{"1min":0.68700000000000006,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.77000000000000002}]},{"osd":11,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.70899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":486300,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.54700000000000004},"min":{"1min":0.38900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78000000000000003,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57999999999999996,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84299999999999997,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54800000000000004,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.54200000000000004,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.153,"5min":1.548,"15min":1.548},"last":0.62}]},{"osd":3,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.432,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Aug 25 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.46700000000000003,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.48099999999999998,"15min":0.42899999999999999},"min":{"1min":0.28399999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.35799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86799999999999999,"5min":1.613,"15min":1.613},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.89700000000000002,"5min":1.597,"15min":1.597},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.437}]},{"osd":7,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.073,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.49099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.38500000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70399999999999996}]},{"osd":9,"last update":"Fri Aug 25 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.628,"5min":0.437,"15min":0.437},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri Aug 25 04:43:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.78500000000000003,"15min":0.76700000000000002},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78700000000000003,"15min":0.76900000000000002},"min":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.66600000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.76100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":484209,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.58399999999999996,"15min":0.51700000000000002},"min":{"1min":0.41699999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.56899999999999995},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":1.194,"15min":1.194},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.877}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.747}]},{"osd":4,"last update":"Fri Aug 25 04:43:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.441,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84999999999999998,"5min":1.488,"15min":1.488},"last":1.167},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.47899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Aug 25 04:42:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46600000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94299999999999995,"5min":1.603,"15min":1.603},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99199999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97499999999999998,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.044},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.121}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.056,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0840000000000001,"5min":1.365,"15min":1.365},"last":0.80500000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0489999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.004}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82699999999999996,"15min":0.81699999999999995},"min":{"1min":0.54700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.1399999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.74299999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0569999999999999,"5min":1.214,"15min":1.214},"last":1.0169999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.92400000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":487923,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.755},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.53000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.436,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.47899999999999998,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.58299999999999996},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Aug 25 04:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.59999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.04},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.014},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.97999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483015,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.39200000000000002},"min":{"1min":0.36899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58199999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.42099999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.64700000000000002,"5min":0.745,"15min":0.745},"last":0.501}]},{"osd":1,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.879},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63900000000000001,"15min":0.58699999999999997},"min":{"1min":0.56100000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.60099999999999998},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89300000000000002,"5min":1.53,"15min":1.53},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":1.573,"15min":1.573},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Aug 25 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65700000000000003,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80900000000000005,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85699999999999998,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.52500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85299999999999998,"5min":1.556,"15min":1.556},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.53100000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.84299999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.747,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.337,"15min":1.337},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:43:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86199999999999999,"15min":0.86799999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.65300000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.274,"15min":1.274},"last":0.93200000000000005}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.88900000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1020000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.83899999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":483375,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.35499999999999998},"min":{"1min":0.34599999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67400000000000004,"5min":0.753,"15min":0.753},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.36899999999999999},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.501}]},{"osd":2,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Aug 25 04:43:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.439,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.99,"15min":1.99},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.45500000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85899999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88800000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.874,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.755}]},{"osd":8,"last update":"Fri Aug 25 04:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93400000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.874}]},{"osd":9,"last update":"Fri Aug 25 04:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.83899999999999997,"15min":0.84899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1180000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92500000000000004,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.97999999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1074575,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.53600000000000003,"15min":0.5},"min":{"1min":0.432,"5min":0.318,"15min":0.318},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.30299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.39400000000000002},"min":{"1min":0.30499999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.40799999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Aug 25 04:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80300000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.52600000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.60799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.495}]},{"osd":9,"last update":"Fri Aug 25 04:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97199999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64400000000000002}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.61599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.72999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:44:03.344 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-08-25T04:44:03.345 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-25T04:44:03.345 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-08-25T04:44:03.345 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph health --format=json 2023-08-25T04:44:03.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:03 smithi162 ceph-mon[130941]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:03.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:03 smithi183 ceph-mon[131414]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:04.067 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:04.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:03 smithi145 ceph-mon[97681]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:04.906 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:04 smithi145 ceph-mon[97681]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:04.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:04 smithi183 ceph-mon[131414]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:05.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:04 smithi162 ceph-mon[130941]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:05.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 -- 172.21.15.145:0/2959592343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174101300 msgr2=0x7fc17410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:05.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 --2- 172.21.15.145:0/2959592343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174101300 0x7fc17410d810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00b0a0 tx=0x7fc16c00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 -- 172.21.15.145:0/2959592343 shutdown_connections 2023-08-25T04:44:05.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 --2- 172.21.15.145:0/2959592343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174101300 0x7fc17410d810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 --2- 172.21.15.145:0/2959592343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174106940 0x7fc174106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 --2- 172.21.15.145:0/2959592343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174100940 0x7fc174100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 -- 172.21.15.145:0/2959592343 >> 172.21.15.145:0/2959592343 conn(0x7fc1740fc410 msgr2=0x7fc1740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:05.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 -- 172.21.15.145:0/2959592343 shutdown_connections 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.600+0000 7fc17a930700 1 -- 172.21.15.145:0/2959592343 wait complete. 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.601+0000 7fc17a930700 1 Processor -- start 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.601+0000 7fc17a930700 1 -- start start 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174101300 0x7fc17419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:05.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174106940 0x7fc174198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741992c0 con 0x7fc174106940 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc174199430 con 0x7fc174100940 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc17a930700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741995f0 con 0x7fc174101300 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc16bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc16bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:59936/0 (socket says 172.21.15.145:59936) 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc16bfff700 1 -- 172.21.15.145:0/780033484 learned_addr learned my addr 172.21.15.145:0/780033484 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:05.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc173fff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174101300 0x7fc17419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc178ecd700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174106940 0x7fc174198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc16bfff700 1 -- 172.21.15.145:0/780033484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174101300 msgr2=0x7fc17419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.602+0000 7fc16bfff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174101300 0x7fc17419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc16bfff700 1 -- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174106940 msgr2=0x7fc174198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc16bfff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174106940 0x7fc174198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc16bfff700 1 -- 172.21.15.145:0/780033484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009d00 con 0x7fc174100940 2023-08-25T04:44:05.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc16bfff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e9a0 tx=0x7fc16400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:05.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164009790 con 0x7fc174100940 2023-08-25T04:44:05.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1741998d0 con 0x7fc174100940 2023-08-25T04:44:05.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc164010430 con 0x7fc174100940 2023-08-25T04:44:05.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164018680 con 0x7fc174100940 2023-08-25T04:44:05.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.603+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1741a5aa0 con 0x7fc174100940 2023-08-25T04:44:05.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.604+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17404e620 con 0x7fc174100940 2023-08-25T04:44:05.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.606+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc1640105a0 con 0x7fc174100940 2023-08-25T04:44:05.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.608+0000 7fc171ffb700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 0x7fc1600a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:05.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.608+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc164014070 con 0x7fc174100940 2023-08-25T04:44:05.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.608+0000 7fc173fff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 0x7fc1600a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:05.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.608+0000 7fc173fff700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 0x7fc1600a2410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc17419a340 tx=0x7fc15c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:05.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.610+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc164095310 con 0x7fc174100940 2023-08-25T04:44:05.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.854+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc174199fd0 con 0x7fc174100940 2023-08-25T04:44:05.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.854+0000 7fc171ffb700 1 -- 172.21.15.145:0/780033484 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc164098930 con 0x7fc174100940 2023-08-25T04:44:05.857 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:05.857 INFO:teuthology.orchestra.run.smithi145.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-25T04:44:05.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 msgr2=0x7fc1600a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:05.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 0x7fc1600a2410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc17419a340 tx=0x7fc15c009260 comp rx=0 tx=0).stop 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 msgr2=0x7fc17419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc16400e9a0 tx=0x7fc16400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 shutdown_connections 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc16009ff50 0x7fc1600a2410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc174106940 0x7fc174198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc174100940 0x7fc17419eb20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 --2- 172.21.15.145:0/780033484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc174101300 0x7fc17419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:05.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 >> 172.21.15.145:0/780033484 conn(0x7fc1740fc410 msgr2=0x7fc17410bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:05.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 shutdown_connections 2023-08-25T04:44:05.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:05.857+0000 7fc17a930700 1 -- 172.21.15.145:0/780033484 wait complete. 2023-08-25T04:44:06.254 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-08-25T04:44:06.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:06.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 -- 172.21.15.145:0/1615351195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c100920 msgr2=0x7ff30c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 --2- 172.21.15.145:0/1615351195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c100920 0x7ff30c100d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff3080099c0 tx=0x7ff308009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 -- 172.21.15.145:0/1615351195 shutdown_connections 2023-08-25T04:44:06.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 --2- 172.21.15.145:0/1615351195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c1024b0 0x7ff30c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 --2- 172.21.15.145:0/1615351195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c100920 0x7ff30c100d40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 --2- 172.21.15.145:0/1615351195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c101b10 0x7ff30c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 -- 172.21.15.145:0/1615351195 >> 172.21.15.145:0/1615351195 conn(0x7ff30c0fbe50 msgr2=0x7ff30c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:06.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 -- 172.21.15.145:0/1615351195 shutdown_connections 2023-08-25T04:44:06.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.458+0000 7ff313dff700 1 -- 172.21.15.145:0/1615351195 wait complete. 2023-08-25T04:44:06.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.459+0000 7ff313dff700 1 Processor -- start 2023-08-25T04:44:06.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.459+0000 7ff313dff700 1 -- start start 2023-08-25T04:44:06.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 0x7ff30c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c1024b0 0x7ff30c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c19e190 con 0x7ff30c101b10 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c19e2d0 con 0x7ff30c100920 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff313dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c19e460 con 0x7ff30c1024b0 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff31139a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 0x7ff30c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:59954/0 (socket says 172.21.15.145:59954) 2023-08-25T04:44:06.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 -- 172.21.15.145:0/4180635424 learned_addr learned my addr 172.21.15.145:0/4180635424 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:06.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff31239c700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c1024b0 0x7ff30c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 -- 172.21.15.145:0/4180635424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c1024b0 msgr2=0x7ff30c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c1024b0 0x7ff30c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 -- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 msgr2=0x7ff30c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 0x7ff30c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff311b9b700 1 -- 172.21.15.145:0/4180635424 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080095f0 con 0x7ff30c100920 2023-08-25T04:44:06.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.460+0000 7ff31139a700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 0x7ff30c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff311b9b700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff308009fd0 tx=0x7ff30800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30801d070 con 0x7ff30c100920 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c19e6e0 con 0x7ff30c100920 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff308005050 con 0x7ff30c100920 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308004a70 con 0x7ff30c100920 2023-08-25T04:44:06.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.461+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c19eb60 con 0x7ff30c100920 2023-08-25T04:44:06.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.462+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c10c180 con 0x7ff30c100920 2023-08-25T04:44:06.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.464+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ff308005450 con 0x7ff30c100920 2023-08-25T04:44:06.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.465+0000 7ff302ffd700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 0x7ff2f80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.465+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff3080cc690 con 0x7ff30c100920 2023-08-25T04:44:06.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.466+0000 7ff31139a700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 0x7ff2f80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.466+0000 7ff31139a700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 0x7ff2f80a2460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc011490 tx=0x7ff2fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:06.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.468+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ff3080961b0 con 0x7ff30c100920 2023-08-25T04:44:06.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.683+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff30c19ed80 con 0x7ff30c100920 2023-08-25T04:44:06.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.686+0000 7ff302ffd700 1 -- 172.21.15.145:0/4180635424 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7ff3080997d0 con 0x7ff30c100920 2023-08-25T04:44:06.687 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:06.687 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:06.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 msgr2=0x7ff2f80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 0x7ff2f80a2460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc011490 tx=0x7ff2fc009260 comp rx=0 tx=0).stop 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 msgr2=0x7ff30c196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff308009fd0 tx=0x7ff30800f690 comp rx=0 tx=0).stop 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 shutdown_connections 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ff2f809ffa0 0x7ff2f80a2460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff30c1024b0 0x7ff30c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff30c100920 0x7ff30c196610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 --2- 172.21.15.145:0/4180635424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff30c101b10 0x7ff30c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 >> 172.21.15.145:0/4180635424 conn(0x7ff30c0fbe50 msgr2=0x7ff30c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:06.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.687+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 shutdown_connections 2023-08-25T04:44:06.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.688+0000 7ff313dff700 1 -- 172.21.15.145:0/4180635424 wait complete. 2023-08-25T04:44:06.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:06.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:06.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 -- 172.21.15.145:0/3050682353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c101b20 msgr2=0x7f232c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 --2- 172.21.15.145:0/3050682353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c101b20 0x7f232c101f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23200099c0 tx=0x7f2320009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 -- 172.21.15.145:0/3050682353 shutdown_connections 2023-08-25T04:44:06.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 --2- 172.21.15.145:0/3050682353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 0x7f232c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 --2- 172.21.15.145:0/3050682353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c100930 0x7f232c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 --2- 172.21.15.145:0/3050682353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c101b20 0x7f232c101f80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 -- 172.21.15.145:0/3050682353 >> 172.21.15.145:0/3050682353 conn(0x7f232c0fbea0 msgr2=0x7f232c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:06.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 -- 172.21.15.145:0/3050682353 shutdown_connections 2023-08-25T04:44:06.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.912+0000 7f2330dff700 1 -- 172.21.15.145:0/3050682353 wait complete. 2023-08-25T04:44:06.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 Processor -- start 2023-08-25T04:44:06.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 -- start start 2023-08-25T04:44:06.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c101b20 0x7f232c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 0x7f232c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0739f0 con 0x7f232c100930 2023-08-25T04:44:06.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c073b60 con 0x7f232c101b20 2023-08-25T04:44:06.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.913+0000 7f2330dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c1a0720 con 0x7f232c1024c0 2023-08-25T04:44:06.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42808/0 (socket says 172.21.15.145:42808) 2023-08-25T04:44:06.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 -- 172.21.15.145:0/1973939611 learned_addr learned my addr 172.21.15.145:0/1973939611 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:06.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f2329d9b700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c101b20 0x7f232c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232ad9d700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 0x7f232c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 -- 172.21.15.145:0/1973939611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 msgr2=0x7f232c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 0x7f232c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 -- 172.21.15.145:0/1973939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c101b20 msgr2=0x7f232c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:06.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c101b20 0x7f232c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:06.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 -- 172.21.15.145:0/1973939611 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23200095f0 con 0x7f232c100930 2023-08-25T04:44:06.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f232a59c700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e970 tx=0x7f231c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:06.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.914+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c009790 con 0x7f232c100930 2023-08-25T04:44:06.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.915+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f231c00eec0 con 0x7f232c100930 2023-08-25T04:44:06.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.915+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c1a0a00 con 0x7f232c100930 2023-08-25T04:44:06.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.915+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c003cf0 con 0x7f232c100930 2023-08-25T04:44:06.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.915+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c1a0f10 con 0x7f232c100930 2023-08-25T04:44:06.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.916+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232c107f10 con 0x7f232c100930 2023-08-25T04:44:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.917+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f231c00ccd0 con 0x7f232c100930 2023-08-25T04:44:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.919+0000 7f231b7fe700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 0x7f23140a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:06.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.919+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f231c014070 con 0x7f232c100930 2023-08-25T04:44:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.919+0000 7f2329d9b700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 0x7f23140a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.920+0000 7f2329d9b700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 0x7f23140a2410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f232000b580 tx=0x7f2320005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:06.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:06.923+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f231c095210 con 0x7f232c100930 2023-08-25T04:44:07.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:06 smithi145 ceph-mon[97681]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:06 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/780033484' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T04:44:07.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:06 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4180635424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:07.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.125+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f232c02cc20 con 0x7f232c100930 2023-08-25T04:44:07.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.126+0000 7f231b7fe700 1 -- 172.21.15.145:0/1973939611 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f231c098830 con 0x7f232c100930 2023-08-25T04:44:07.128 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:07.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 msgr2=0x7f23140a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 0x7f23140a2410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f232000b580 tx=0x7f2320005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 msgr2=0x7f232c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e970 tx=0x7f231c00ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:07.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 shutdown_connections 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f231409ff50 0x7f23140a2410 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f232c1024c0 0x7f232c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f232c100930 0x7f232c0746d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 --2- 172.21.15.145:0/1973939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f232c101b20 0x7f232c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 >> 172.21.15.145:0/1973939611 conn(0x7f232c0fbea0 msgr2=0x7f232c105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.128+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 shutdown_connections 2023-08-25T04:44:07.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.129+0000 7f2330dff700 1 -- 172.21.15.145:0/1973939611 wait complete. 2023-08-25T04:44:07.149 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:07.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:06 smithi162 ceph-mon[130941]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:07.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:06 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/780033484' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T04:44:07.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:06 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4180635424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:07.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:06 smithi183 ceph-mon[131414]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:07.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:06 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/780033484' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-25T04:44:07.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:06 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4180635424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:07.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 -- 172.21.15.145:0/2031493943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b4100930 msgr2=0x7f88b4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/2031493943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b4100930 0x7f88b4100d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f88a80099c0 tx=0x7f88a8009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 -- 172.21.15.145:0/2031493943 shutdown_connections 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/2031493943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b41024c0 0x7f88b4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/2031493943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b4100930 0x7f88b4100d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/2031493943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 0x7f88b4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.349+0000 7f88bb1ff700 1 -- 172.21.15.145:0/2031493943 >> 172.21.15.145:0/2031493943 conn(0x7f88b40fbea0 msgr2=0x7f88b40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.350+0000 7f88bb1ff700 1 -- 172.21.15.145:0/2031493943 shutdown_connections 2023-08-25T04:44:07.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.350+0000 7f88bb1ff700 1 -- 172.21.15.145:0/2031493943 wait complete. 2023-08-25T04:44:07.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 Processor -- start 2023-08-25T04:44:07.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 -- start start 2023-08-25T04:44:07.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4100930 0x7f88b4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 0x7f88b4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41a03a0 con 0x7f88b41024c0 2023-08-25T04:44:07.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41a04e0 con 0x7f88b4101b20 2023-08-25T04:44:07.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88bb1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41a0670 con 0x7f88b4100930 2023-08-25T04:44:07.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88b979c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88b979c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42816/0 (socket says 172.21.15.145:42816) 2023-08-25T04:44:07.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88b979c700 1 -- 172.21.15.145:0/3349530867 learned_addr learned my addr 172.21.15.145:0/3349530867 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:07.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.351+0000 7f88b3fff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 0x7f88b4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b8f9b700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4100930 0x7f88b4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 -- 172.21.15.145:0/3349530867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4100930 msgr2=0x7f88b4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4100930 0x7f88b4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 -- 172.21.15.145:0/3349530867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 msgr2=0x7f88b4198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 0x7f88b4198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 -- 172.21.15.145:0/3349530867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80095f0 con 0x7f88b41024c0 2023-08-25T04:44:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b979c700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00ef30 tx=0x7f88ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac018070 con 0x7f88b41024c0 2023-08-25T04:44:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ac00f040 con 0x7f88b41024c0 2023-08-25T04:44:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b41a08f0 con 0x7f88b41024c0 2023-08-25T04:44:07.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00cd00 con 0x7f88b41024c0 2023-08-25T04:44:07.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.352+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b41a0db0 con 0x7f88b41024c0 2023-08-25T04:44:07.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.353+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b4192f30 con 0x7f88b41024c0 2023-08-25T04:44:07.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.355+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f88ac003a00 con 0x7f88b41024c0 2023-08-25T04:44:07.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.357+0000 7f88b1ffb700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 0x7f88a40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.357+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f88ac0cb0f0 con 0x7f88b41024c0 2023-08-25T04:44:07.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.357+0000 7f88b8f9b700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 0x7f88a40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.358+0000 7f88b8f9b700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 0x7f88a40a2410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009fd0 tx=0x7f88a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:07.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.361+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f88ac094bb0 con 0x7f88b41024c0 2023-08-25T04:44:07.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.562+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88b402cde0 con 0x7f88b41024c0 2023-08-25T04:44:07.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.563+0000 7f88b1ffb700 1 -- 172.21.15.145:0/3349530867 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88ac0981d0 con 0x7f88b41024c0 2023-08-25T04:44:07.565 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:07.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 msgr2=0x7f88a40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 0x7f88a40a2410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009fd0 tx=0x7f88a8005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 msgr2=0x7f88b419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00ef30 tx=0x7f88ac00c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 shutdown_connections 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f88a409ff50 0x7f88a40a2410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b41024c0 0x7f88b419fdd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4100930 0x7f88b4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 --2- 172.21.15.145:0/3349530867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b4101b20 0x7f88b4198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.565+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 >> 172.21.15.145:0/3349530867 conn(0x7f88b40fbea0 msgr2=0x7f88b4107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.566+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 shutdown_connections 2023-08-25T04:44:07.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.566+0000 7f88bb1ff700 1 -- 172.21.15.145:0/3349530867 wait complete. 2023-08-25T04:44:07.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:07.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 -- 172.21.15.145:0/3577000046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c1024c0 msgr2=0x7ffb5c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 --2- 172.21.15.145:0/3577000046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c108d70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400b0d0 tx=0x7ffb5400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 -- 172.21.15.145:0/3577000046 shutdown_connections 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 --2- 172.21.15.145:0/3577000046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c108d70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 --2- 172.21.15.145:0/3577000046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c100930 0x7ffb5c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 --2- 172.21.15.145:0/3577000046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c101b20 0x7ffb5c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.788+0000 7ffb611ff700 1 -- 172.21.15.145:0/3577000046 >> 172.21.15.145:0/3577000046 conn(0x7ffb5c0fbea0 msgr2=0x7ffb5c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.789+0000 7ffb611ff700 1 -- 172.21.15.145:0/3577000046 shutdown_connections 2023-08-25T04:44:07.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.789+0000 7ffb611ff700 1 -- 172.21.15.145:0/3577000046 wait complete. 2023-08-25T04:44:07.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 Processor -- start 2023-08-25T04:44:07.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 -- start start 2023-08-25T04:44:07.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c101b20 0x7ffb5c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1a04b0 con 0x7ffb5c100930 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1a05f0 con 0x7ffb5c1024c0 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb611ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1a0780 con 0x7ffb5c101b20 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42832/0 (socket says 172.21.15.145:42832) 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb5ad9d700 1 -- 172.21.15.145:0/2538786979 learned_addr learned my addr 172.21.15.145:0/2538786979 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb5a59c700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c101b20 0x7ffb5c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.790+0000 7ffb5b59e700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 -- 172.21.15.145:0/2538786979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c101b20 msgr2=0x7ffb5c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c101b20 0x7ffb5c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 -- 172.21.15.145:0/2538786979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 msgr2=0x7ffb5c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 -- 172.21.15.145:0/2538786979 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54009d00 con 0x7ffb5c100930 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5b59e700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb5ad9d700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e970 tx=0x7ffb4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c009790 con 0x7ffb5c100930 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb4c00eec0 con 0x7ffb5c100930 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c1a0a60 con 0x7ffb5c100930 2023-08-25T04:44:07.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c003cf0 con 0x7ffb5c100930 2023-08-25T04:44:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.791+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c1a0fc0 con 0x7ffb5c100930 2023-08-25T04:44:07.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.792+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c10c1a0 con 0x7ffb5c100930 2023-08-25T04:44:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.794+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7ffb4c00ccd0 con 0x7ffb5c100930 2023-08-25T04:44:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.796+0000 7ffb43fff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 0x7ffb440a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.797+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ffb4c014070 con 0x7ffb5c100930 2023-08-25T04:44:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.797+0000 7ffb5a59c700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 0x7ffb440a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:07.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.797+0000 7ffb5a59c700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 0x7ffb440a2490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffb50007be0 tx=0x7ffb500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:07.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:07.800+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7ffb4c095180 con 0x7ffb5c100930 2023-08-25T04:44:08.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.003+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb5c1a0bf0 con 0x7ffb5c100930 2023-08-25T04:44:08.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.004+0000 7ffb43fff700 1 -- 172.21.15.145:0/2538786979 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb5c1a0bf0 con 0x7ffb5c100930 2023-08-25T04:44:08.006 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:08.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 msgr2=0x7ffb440a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 0x7ffb440a2490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffb50007be0 tx=0x7ffb500073d0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 msgr2=0x7ffb5c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e970 tx=0x7ffb4c00ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 shutdown_connections 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7ffb4409ffd0 0x7ffb440a2490 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5c1024c0 0x7ffb5c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb5c100930 0x7ffb5c198930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 --2- 172.21.15.145:0/2538786979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb5c101b20 0x7ffb5c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.006+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 >> 172.21.15.145:0/2538786979 conn(0x7ffb5c0fbea0 msgr2=0x7ffb5c107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:08.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.007+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 shutdown_connections 2023-08-25T04:44:08.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.007+0000 7ffb611ff700 1 -- 172.21.15.145:0/2538786979 wait complete. 2023-08-25T04:44:08.025 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:07 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1973939611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.025 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:07 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3349530867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.027 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:07 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1973939611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:07 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3349530867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:07 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1973939611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:07 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3349530867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:08.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 -- 172.21.15.145:0/1070899676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc100930 msgr2=0x7f3ccc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/1070899676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc100930 0x7f3ccc100d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0099c0 tx=0x7f3cbc009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 -- 172.21.15.145:0/1070899676 shutdown_connections 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/1070899676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/1070899676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc100930 0x7f3ccc100d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/1070899676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc101b20 0x7f3ccc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 -- 172.21.15.145:0/1070899676 >> 172.21.15.145:0/1070899676 conn(0x7f3ccc0fbea0 msgr2=0x7f3ccc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 -- 172.21.15.145:0/1070899676 shutdown_connections 2023-08-25T04:44:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.226+0000 7f3cd13ff700 1 -- 172.21.15.145:0/1070899676 wait complete. 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.227+0000 7f3cd13ff700 1 Processor -- start 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.227+0000 7f3cd13ff700 1 -- start start 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc100930 0x7f3ccc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc101b20 0x7f3ccc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a03f0 con 0x7f3ccc1024c0 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a0530 con 0x7f3ccc101b20 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cd13ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1a06c0 con 0x7f3ccc100930 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42856/0 (socket says 172.21.15.145:42856) 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 -- 172.21.15.145:0/309371179 learned_addr learned my addr 172.21.15.145:0/309371179 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccaffd700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc100930 0x7f3ccc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3cca7fc700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc101b20 0x7f3ccc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 -- 172.21.15.145:0/309371179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc100930 msgr2=0x7f3ccc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc100930 0x7f3ccc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 -- 172.21.15.145:0/309371179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc101b20 msgr2=0x7f3ccc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc101b20 0x7f3ccc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.228+0000 7f3ccb7fe700 1 -- 172.21.15.145:0/309371179 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc0095f0 con 0x7f3ccc1024c0 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3ccb7fe700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400ef30 tx=0x7f3cc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:08.233 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4018070 con 0x7f3ccc1024c0 2023-08-25T04:44:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc400f040 con 0x7f3ccc1024c0 2023-08-25T04:44:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc1a0940 con 0x7f3ccc1024c0 2023-08-25T04:44:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400cd00 con 0x7f3ccc1024c0 2023-08-25T04:44:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.229+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc1a0e50 con 0x7f3ccc1024c0 2023-08-25T04:44:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.230+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc192f80 con 0x7f3ccc1024c0 2023-08-25T04:44:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.232+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3cc4003a00 con 0x7f3ccc1024c0 2023-08-25T04:44:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.233+0000 7f3cb3fff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 0x7f3cb40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.233+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3cc40cb080 con 0x7f3ccc1024c0 2023-08-25T04:44:08.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.234+0000 7f3ccaffd700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 0x7f3cb40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.234+0000 7f3ccaffd700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 0x7f3cb40a2490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009990 tx=0x7f3cbc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:08.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.237+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3cc40fde00 con 0x7f3ccc1024c0 2023-08-25T04:44:08.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.438+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3ccc02cd50 con 0x7f3ccc1024c0 2023-08-25T04:44:08.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.439+0000 7f3cb3fff700 1 -- 172.21.15.145:0/309371179 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3cc4094b40 con 0x7f3ccc1024c0 2023-08-25T04:44:08.441 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:08.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.441+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 msgr2=0x7f3cb40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.441+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 0x7f3cb40a2490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009990 tx=0x7f3cbc005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 msgr2=0x7f3ccc19fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400ef30 tx=0x7f3cc400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 shutdown_connections 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3cb409ffd0 0x7f3cb40a2490 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ccc1024c0 0x7f3ccc19fe20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ccc100930 0x7f3ccc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 --2- 172.21.15.145:0/309371179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ccc101b20 0x7f3ccc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 >> 172.21.15.145:0/309371179 conn(0x7f3ccc0fbea0 msgr2=0x7f3ccc107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:08.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 shutdown_connections 2023-08-25T04:44:08.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.442+0000 7f3cd13ff700 1 -- 172.21.15.145:0/309371179 wait complete. 2023-08-25T04:44:08.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:08.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 -- 172.21.15.145:0/3316570624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728101b20 msgr2=0x7fd728101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 --2- 172.21.15.145:0/3316570624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728101b20 0x7fd728101f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0098f0 tx=0x7fd71c009c00 comp rx=0 tx=0).stop 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 -- 172.21.15.145:0/3316570624 shutdown_connections 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 --2- 172.21.15.145:0/3316570624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7281024c0 0x7fd728108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 --2- 172.21.15.145:0/3316570624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd728100930 0x7fd728100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 --2- 172.21.15.145:0/3316570624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728101b20 0x7fd728101f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 -- 172.21.15.145:0/3316570624 >> 172.21.15.145:0/3316570624 conn(0x7fd7280fbe60 msgr2=0x7fd7280fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.659+0000 7fd72ddff700 1 -- 172.21.15.145:0/3316570624 shutdown_connections 2023-08-25T04:44:08.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.660+0000 7fd72ddff700 1 -- 172.21.15.145:0/3316570624 wait complete. 2023-08-25T04:44:08.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.661+0000 7fd72ddff700 1 Processor -- start 2023-08-25T04:44:08.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.661+0000 7fd72ddff700 1 -- start start 2023-08-25T04:44:08.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728100930 0x7fd728196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 0x7fd72819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819e1d0 con 0x7fd728101b20 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819e310 con 0x7fd7281024c0 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd72ddff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819e4a0 con 0x7fd728100930 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42866/0 (socket says 172.21.15.145:42866) 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd727fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 0x7fd72819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 -- 172.21.15.145:0/851413858 learned_addr learned my addr 172.21.15.145:0/851413858 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:08.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd7277fe700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728100930 0x7fd728196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 -- 172.21.15.145:0/851413858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728100930 msgr2=0x7fd728196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728100930 0x7fd728196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 -- 172.21.15.145:0/851413858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 msgr2=0x7fd72819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 0x7fd72819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd726ffd700 1 -- 172.21.15.145:0/851413858 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c009520 con 0x7fd728101b20 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.662+0000 7fd727fff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 0x7fd72819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:08.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd726ffd700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fd0 tx=0x7fd71c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:08.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c01c070 con 0x7fd728101b20 2023-08-25T04:44:08.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71c00bc10 con 0x7fd728101b20 2023-08-25T04:44:08.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c004a20 con 0x7fd728101b20 2023-08-25T04:44:08.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72819e720 con 0x7fd728101b20 2023-08-25T04:44:08.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.663+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72819eaf0 con 0x7fd728101b20 2023-08-25T04:44:08.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.664+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd728107ef0 con 0x7fd728101b20 2023-08-25T04:44:08.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.666+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd71c020070 con 0x7fd728101b20 2023-08-25T04:44:08.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.667+0000 7fd724ff9700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 0x7fd7100a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:08.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.668+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd71c0cc2f0 con 0x7fd728101b20 2023-08-25T04:44:08.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.668+0000 7fd7277fe700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 0x7fd7100a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:08.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.668+0000 7fd7277fe700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 0x7fd7100a2460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd728101980 tx=0x7fd718009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.670+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd71c095eb0 con 0x7fd728101b20 2023-08-25T04:44:08.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.923+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd72810c3e0 con 0x7fd728101b20 2023-08-25T04:44:08.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.924+0000 7fd724ff9700 1 -- 172.21.15.145:0/851413858 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7fd71c0994d0 con 0x7fd728101b20 2023-08-25T04:44:08.926 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:08.926 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:08.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 msgr2=0x7fd7100a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 0x7fd7100a2460 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd728101980 tx=0x7fd718009260 comp rx=0 tx=0).stop 2023-08-25T04:44:08.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 msgr2=0x7fd728196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:08.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fd0 tx=0x7fd71c00b560 comp rx=0 tx=0).stop 2023-08-25T04:44:08.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 shutdown_connections 2023-08-25T04:44:08.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd71009ffa0 0x7fd7100a2460 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7281024c0 0x7fd72819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728100930 0x7fd728196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 --2- 172.21.15.145:0/851413858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd728101b20 0x7fd728196b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:08.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.926+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 >> 172.21.15.145:0/851413858 conn(0x7fd7280fbe60 msgr2=0x7fd728105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:08.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.927+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 shutdown_connections 2023-08-25T04:44:08.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:08.927+0000 7fd72ddff700 1 -- 172.21.15.145:0/851413858 wait complete. 2023-08-25T04:44:08.932 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:08.947 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:09.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:08 smithi145 ceph-mon[97681]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2538786979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:08 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/309371179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 -- 172.21.15.145:0/3973558310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9940101b20 msgr2=0x7f9940101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 --2- 172.21.15.145:0/3973558310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9940101b20 0x7f9940101f80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f99340099c0 tx=0x7f9934009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 -- 172.21.15.145:0/3973558310 shutdown_connections 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 --2- 172.21.15.145:0/3973558310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99401024c0 0x7f9940108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 --2- 172.21.15.145:0/3973558310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940100930 0x7f9940100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 --2- 172.21.15.145:0/3973558310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9940101b20 0x7f9940101f80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 -- 172.21.15.145:0/3973558310 >> 172.21.15.145:0/3973558310 conn(0x7f99400fbe60 msgr2=0x7f99400fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:09.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 -- 172.21.15.145:0/3973558310 shutdown_connections 2023-08-25T04:44:09.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.144+0000 7f99451ff700 1 -- 172.21.15.145:0/3973558310 wait complete. 2023-08-25T04:44:09.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 Processor -- start 2023-08-25T04:44:09.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 -- start start 2023-08-25T04:44:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9940100930 0x7f99401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 0x7f9940198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a0370 con 0x7f99401024c0 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a04b0 con 0x7f9940101b20 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f99451ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a0640 con 0x7f9940100930 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.145+0000 7f993f59e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993f59e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42880/0 (socket says 172.21.15.145:42880) 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 0x7f9940198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993ed9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9940100930 0x7f99401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993f59e700 1 -- 172.21.15.145:0/2288960808 learned_addr learned my addr 172.21.15.145:0/2288960808 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 -- 172.21.15.145:0/2288960808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9940100930 msgr2=0x7f99401987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9940100930 0x7f99401987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 -- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 msgr2=0x7f994019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 -- 172.21.15.145:0/2288960808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340095f0 con 0x7f9940101b20 2023-08-25T04:44:09.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993f59e700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:09.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f993e59c700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 0x7f9940198d30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99340098e0 tx=0x7f993400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:09.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.146+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401d070 con 0x7f9940101b20 2023-08-25T04:44:09.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.147+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9934005050 con 0x7f9940101b20 2023-08-25T04:44:09.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.147+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401a08c0 con 0x7f9940101b20 2023-08-25T04:44:09.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.147+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934004ae0 con 0x7f9940101b20 2023-08-25T04:44:09.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.147+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401a0ce0 con 0x7f9940101b20 2023-08-25T04:44:09.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.148+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9940192f80 con 0x7f9940101b20 2023-08-25T04:44:09.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.150+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9934005450 con 0x7f9940101b20 2023-08-25T04:44:09.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.151+0000 7f9927fff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 0x7f99280a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.151+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f99340cc770 con 0x7f9940101b20 2023-08-25T04:44:09.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.151+0000 7f993ed9d700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 0x7f99280a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.152+0000 7f993ed9d700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 0x7f99280a2460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9940101980 tx=0x7f9930009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:09.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.153+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9934096230 con 0x7f9940101b20 2023-08-25T04:44:09.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:08 smithi162 ceph-mon[130941]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2538786979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:08 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/309371179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:08 smithi183 ceph-mon[131414]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:09.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2538786979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:08 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/309371179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.357+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f994010c360 con 0x7f9940101b20 2023-08-25T04:44:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.359+0000 7f9927fff700 1 -- 172.21.15.145:0/2288960808 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f9934099850 con 0x7f9940101b20 2023-08-25T04:44:09.361 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:09.362 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:09.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 msgr2=0x7f99280a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 0x7f99280a2460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9940101980 tx=0x7f9930009260 comp rx=0 tx=0).stop 2023-08-25T04:44:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 msgr2=0x7f9940198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 0x7f9940198d30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99340098e0 tx=0x7f993400f690 comp rx=0 tx=0).stop 2023-08-25T04:44:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 shutdown_connections 2023-08-25T04:44:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f992809ffa0 0x7f99280a2460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99401024c0 0x7f994019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9940100930 0x7f99401987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 --2- 172.21.15.145:0/2288960808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9940101b20 0x7f9940198d30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 >> 172.21.15.145:0/2288960808 conn(0x7f99400fbe60 msgr2=0x7f99401073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.361+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 shutdown_connections 2023-08-25T04:44:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.362+0000 7f99451ff700 1 -- 172.21.15.145:0/2288960808 wait complete. 2023-08-25T04:44:09.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:09.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:09.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.582+0000 7faaac1ff700 1 -- 172.21.15.145:0/3978743662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4100910 msgr2=0x7faaa4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.582+0000 7faaac1ff700 1 --2- 172.21.15.145:0/3978743662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4100910 0x7faaa4100d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faaa00099c0 tx=0x7faaa0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 -- 172.21.15.145:0/3978743662 shutdown_connections 2023-08-25T04:44:09.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 --2- 172.21.15.145:0/3978743662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa41024a0 0x7faaa4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 --2- 172.21.15.145:0/3978743662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4100910 0x7faaa4100d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 --2- 172.21.15.145:0/3978743662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa4101b00 0x7faaa4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 -- 172.21.15.145:0/3978743662 >> 172.21.15.145:0/3978743662 conn(0x7faaa40fbe60 msgr2=0x7faaa40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:09.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 -- 172.21.15.145:0/3978743662 shutdown_connections 2023-08-25T04:44:09.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.583+0000 7faaac1ff700 1 -- 172.21.15.145:0/3978743662 wait complete. 2023-08-25T04:44:09.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 Processor -- start 2023-08-25T04:44:09.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 -- start start 2023-08-25T04:44:09.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa4100910 0x7faaa4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa41024a0 0x7faaa419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41a0400 con 0x7faaa4101b00 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41a0540 con 0x7faaa4100910 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaac1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41a06d0 con 0x7faaa41024a0 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaa979a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaa979a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42888/0 (socket says 172.21.15.145:42888) 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.584+0000 7faaa979a700 1 -- 172.21.15.145:0/837820169 learned_addr learned my addr 172.21.15.145:0/837820169 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:09.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaaa79c700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa41024a0 0x7faaa419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa9f9b700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa4100910 0x7faaa4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 -- 172.21.15.145:0/837820169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa41024a0 msgr2=0x7faaa419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa41024a0 0x7faaa419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 -- 172.21.15.145:0/837820169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa4100910 msgr2=0x7faaa4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa4100910 0x7faaa4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 -- 172.21.15.145:0/837820169 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa00095f0 con 0x7faaa4101b00 2023-08-25T04:44:09.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.585+0000 7faaa979a700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa9400d950 tx=0x7faa9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.586+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400f890 con 0x7faaa4101b00 2023-08-25T04:44:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.586+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa41a09b0 con 0x7faaa4101b00 2023-08-25T04:44:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.586+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa94010460 con 0x7faaa4101b00 2023-08-25T04:44:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.586+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400e610 con 0x7faaa4101b00 2023-08-25T04:44:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.586+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa41a0f10 con 0x7faaa4101b00 2023-08-25T04:44:09.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.587+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4192f60 con 0x7faaa4101b00 2023-08-25T04:44:09.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.589+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7faa940108d0 con 0x7faaa4101b00 2023-08-25T04:44:09.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.591+0000 7faa9affd700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 0x7faa900a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:09.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.591+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faa940cb330 con 0x7faaa4101b00 2023-08-25T04:44:09.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.591+0000 7faaa9f9b700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 0x7faa900a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:09.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.592+0000 7faaa9f9b700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 0x7faa900a2410 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009990 tx=0x7faaa0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:09.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.595+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faa94094d60 con 0x7faaa4101b00 2023-08-25T04:44:09.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.796+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faaa402cd50 con 0x7faaa4101b00 2023-08-25T04:44:09.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.796+0000 7faa9affd700 1 -- 172.21.15.145:0/837820169 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa94098380 con 0x7faaa4101b00 2023-08-25T04:44:09.798 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:09.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.798+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 msgr2=0x7faa900a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.798+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 0x7faa900a2410 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009990 tx=0x7faaa0005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.798+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 msgr2=0x7faaa4198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.798+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa9400d950 tx=0x7faa9400dc60 comp rx=0 tx=0).stop 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 shutdown_connections 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faa9009ff50 0x7faa900a2410 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaa41024a0 0x7faaa419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa4100910 0x7faaa4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 --2- 172.21.15.145:0/837820169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4101b00 0x7faaa4198dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 >> 172.21.15.145:0/837820169 conn(0x7faaa40fbe60 msgr2=0x7faaa4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:09.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 shutdown_connections 2023-08-25T04:44:09.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:09.799+0000 7faaac1ff700 1 -- 172.21.15.145:0/837820169 wait complete. 2023-08-25T04:44:09.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 -- 172.21.15.145:0/2260807675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 msgr2=0x7f83d0108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 --2- 172.21.15.145:0/2260807675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d0108d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b0d0 tx=0x7f83c800b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 -- 172.21.15.145:0/2260807675 shutdown_connections 2023-08-25T04:44:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 --2- 172.21.15.145:0/2260807675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d0108d70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 --2- 172.21.15.145:0/2260807675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0100930 0x7f83d0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 --2- 172.21.15.145:0/2260807675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0101b20 0x7f83d0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 -- 172.21.15.145:0/2260807675 >> 172.21.15.145:0/2260807675 conn(0x7f83d00fbea0 msgr2=0x7f83d00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 -- 172.21.15.145:0/2260807675 shutdown_connections 2023-08-25T04:44:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.019+0000 7f83d81ff700 1 -- 172.21.15.145:0/2260807675 wait complete. 2023-08-25T04:44:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 Processor -- start 2023-08-25T04:44:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 -- start start 2023-08-25T04:44:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0101b20 0x7f83d0196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d019e2a0 con 0x7f83d01024c0 2023-08-25T04:44:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d019e3e0 con 0x7f83d0100930 2023-08-25T04:44:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.020+0000 7f83d81ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d019e570 con 0x7f83d0101b20 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d679c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60094/0 (socket says 172.21.15.145:60094) 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 -- 172.21.15.145:0/3155812482 learned_addr learned my addr 172.21.15.145:0/3155812482 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d579a700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0101b20 0x7f83d0196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 -- 172.21.15.145:0/3155812482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0101b20 msgr2=0x7f83d0196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0101b20 0x7f83d0196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 -- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 msgr2=0x7f83d019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 -- 172.21.15.145:0/3155812482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8009d00 con 0x7f83d0100930 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d679c700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83d5f9b700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e970 tx=0x7f83cc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.021+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc009790 con 0x7f83d0100930 2023-08-25T04:44:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.022+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d019e850 con 0x7f83d0100930 2023-08-25T04:44:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.022+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83cc00c7e0 con 0x7f83d0100930 2023-08-25T04:44:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.022+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc003c70 con 0x7f83d0100930 2023-08-25T04:44:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.022+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d019edb0 con 0x7f83d0100930 2023-08-25T04:44:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.023+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d0107e20 con 0x7f83d0100930 2023-08-25T04:44:10.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.025+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f83cc00cc50 con 0x7f83d0100930 2023-08-25T04:44:10.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.026+0000 7f83c6ffd700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 0x7f83bc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.026+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f83cc014070 con 0x7f83d0100930 2023-08-25T04:44:10.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.026+0000 7f83d579a700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 0x7f83bc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.027+0000 7f83d579a700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 0x7f83bc0a2410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009c60 tx=0x7f83c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.030+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f83cc095150 con 0x7f83d0100930 2023-08-25T04:44:10.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:09 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/851413858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:09 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2288960808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:09 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/851413858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:09 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2288960808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:09 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/851413858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:09 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2288960808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:10.234 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.232+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83d010c1a0 con 0x7f83d0100930 2023-08-25T04:44:10.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.233+0000 7f83c6ffd700 1 -- 172.21.15.145:0/3155812482 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83d010c1a0 con 0x7f83d0100930 2023-08-25T04:44:10.235 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:10.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 msgr2=0x7f83bc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 0x7f83bc0a2410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009c60 tx=0x7f83c0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 msgr2=0x7f83d0196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e970 tx=0x7f83cc00ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 shutdown_connections 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f83bc09ff50 0x7f83bc0a2410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01024c0 0x7f83d019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d0100930 0x7f83d0196720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 --2- 172.21.15.145:0/3155812482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83d0101b20 0x7f83d0196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 >> 172.21.15.145:0/3155812482 conn(0x7f83d00fbea0 msgr2=0x7f83d0104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:10.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 shutdown_connections 2023-08-25T04:44:10.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.236+0000 7f83d81ff700 1 -- 172.21.15.145:0/3155812482 wait complete. 2023-08-25T04:44:10.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:10.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 -- 172.21.15.145:0/2101877375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe448074960 msgr2=0x7fe448072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 --2- 172.21.15.145:0/2101877375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe448074960 0x7fe448072db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009990 tx=0x7fe43c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 -- 172.21.15.145:0/2101877375 shutdown_connections 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 --2- 172.21.15.145:0/2101877375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4481087d0 0x7fe44810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 --2- 172.21.15.145:0/2101877375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe448074960 0x7fe448072db0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 --2- 172.21.15.145:0/2101877375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448073380 0x7fe448073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 -- 172.21.15.145:0/2101877375 >> 172.21.15.145:0/2101877375 conn(0x7fe4480fbe60 msgr2=0x7fe4480fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 -- 172.21.15.145:0/2101877375 shutdown_connections 2023-08-25T04:44:10.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.457+0000 7fe44edff700 1 -- 172.21.15.145:0/2101877375 wait complete. 2023-08-25T04:44:10.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.458+0000 7fe44edff700 1 Processor -- start 2023-08-25T04:44:10.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.458+0000 7fe44edff700 1 -- start start 2023-08-25T04:44:10.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe448073380 0x7fe448103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448074960 0x7fe448104170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448100f60 con 0x7fe4481087d0 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481010d0 con 0x7fe448073380 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44edff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448101290 con 0x7fe448074960 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42930/0 (socket says 172.21.15.145:42930) 2023-08-25T04:44:10.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 -- 172.21.15.145:0/4046953798 learned_addr learned my addr 172.21.15.145:0/4046953798 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44cb9b700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe448073380 0x7fe448103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe447fff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448074960 0x7fe448104170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 -- 172.21.15.145:0/4046953798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448074960 msgr2=0x7fe448104170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448074960 0x7fe448104170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 -- 172.21.15.145:0/4046953798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe448073380 msgr2=0x7fe448103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe448073380 0x7fe448103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.459+0000 7fe44d39c700 1 -- 172.21.15.145:0/4046953798 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0095f0 con 0x7fe4481087d0 2023-08-25T04:44:10.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe44d39c700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe440007ee0 tx=0x7fe44000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440017070 con 0x7fe4481087d0 2023-08-25T04:44:10.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4481a4dc0 con 0x7fe4481087d0 2023-08-25T04:44:10.465 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44000f040 con 0x7fe4481087d0 2023-08-25T04:44:10.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440005580 con 0x7fe4481087d0 2023-08-25T04:44:10.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.460+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4481a51f0 con 0x7fe4481087d0 2023-08-25T04:44:10.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.461+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe448106e80 con 0x7fe4481087d0 2023-08-25T04:44:10.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.463+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fe44000db20 con 0x7fe4481087d0 2023-08-25T04:44:10.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.464+0000 7fe445ffb700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 0x7fe4380a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.464+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe4400caef0 con 0x7fe4481087d0 2023-08-25T04:44:10.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.465+0000 7fe44cb9b700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 0x7fe4380a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.465+0000 7fe44cb9b700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 0x7fe4380a2410 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00b2e0 tx=0x7fe43c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.466+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe440094920 con 0x7fe4481087d0 2023-08-25T04:44:10.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.669+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe44810e5a0 con 0x7fe4481087d0 2023-08-25T04:44:10.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.670+0000 7fe445ffb700 1 -- 172.21.15.145:0/4046953798 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe440097f40 con 0x7fe4481087d0 2023-08-25T04:44:10.672 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 msgr2=0x7fe4380a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 0x7fe4380a2410 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00b2e0 tx=0x7fe43c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 msgr2=0x7fe4481007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe440007ee0 tx=0x7fe44000d3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 shutdown_connections 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fe43809ff50 0x7fe4380a2410 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4481087d0 0x7fe4481007c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe448073380 0x7fe448103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 --2- 172.21.15.145:0/4046953798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe448074960 0x7fe448104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.672+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 >> 172.21.15.145:0/4046953798 conn(0x7fe4480fbe60 msgr2=0x7fe4480fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.673+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 shutdown_connections 2023-08-25T04:44:10.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.673+0000 7fe44edff700 1 -- 172.21.15.145:0/4046953798 wait complete. 2023-08-25T04:44:10.693 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 -- 172.21.15.145:0/1229756208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 msgr2=0x7f6180108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 --2- 172.21.15.145:0/1229756208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180108d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f617800b0a0 tx=0x7f617800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 -- 172.21.15.145:0/1229756208 shutdown_connections 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 --2- 172.21.15.145:0/1229756208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180108d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 --2- 172.21.15.145:0/1229756208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180100910 0x7f6180100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 --2- 172.21.15.145:0/1229756208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180101b00 0x7f6180101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 -- 172.21.15.145:0/1229756208 >> 172.21.15.145:0/1229756208 conn(0x7f61800fbe60 msgr2=0x7f61800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:10.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 -- 172.21.15.145:0/1229756208 shutdown_connections 2023-08-25T04:44:10.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.893+0000 7f61887ff700 1 -- 172.21.15.145:0/1229756208 wait complete. 2023-08-25T04:44:10.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 Processor -- start 2023-08-25T04:44:10.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 -- start start 2023-08-25T04:44:10.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180100910 0x7f61800746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61800739f0 con 0x7f6180101b00 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180073b60 con 0x7f6180100910 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.894+0000 7f61887ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61801a07e0 con 0x7f61801024a0 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42954/0 (socket says 172.21.15.145:42954) 2023-08-25T04:44:10.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 -- 172.21.15.145:0/4271679321 learned_addr learned my addr 172.21.15.145:0/4271679321 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f618659b700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180100910 0x7f61800746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6186d9c700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 -- 172.21.15.145:0/4271679321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 msgr2=0x7f6180073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 -- 172.21.15.145:0/4271679321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180100910 msgr2=0x7f61800746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:10.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180100910 0x7f61800746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 -- 172.21.15.145:0/4271679321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c009520 con 0x7f6180101b00 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.895+0000 7f6185d9a700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f617000e9c0 tx=0x7f617000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.896+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700097b0 con 0x7f6180101b00 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.896+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6178009d00 con 0x7f6180101b00 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.896+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617000ee60 con 0x7f6180101b00 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.896+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700054f0 con 0x7f6180101b00 2023-08-25T04:44:10.900 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.896+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61801a0e10 con 0x7f6180101b00 2023-08-25T04:44:10.901 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.897+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618010c1a0 con 0x7f6180101b00 2023-08-25T04:44:10.901 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.899+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f6170004750 con 0x7f6180101b00 2023-08-25T04:44:10.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.900+0000 7f61777fe700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 0x7f616c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:10.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.900+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6170014070 con 0x7f6180101b00 2023-08-25T04:44:10.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.900+0000 7f618659b700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 0x7f616c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:10.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.901+0000 7f618659b700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 0x7f616c0a2410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6180101960 tx=0x7f617c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:10.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:10.902+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f61700953a0 con 0x7f6180101b00 2023-08-25T04:44:11.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:10 smithi145 ceph-mon[97681]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:11.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/837820169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3155812482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:10 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4046953798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.103+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f618010c390 con 0x7f6180101b00 2023-08-25T04:44:11.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.104+0000 7f61777fe700 1 -- 172.21.15.145:0/4271679321 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f61700989c0 con 0x7f6180101b00 2023-08-25T04:44:11.107 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:11.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 msgr2=0x7f616c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 0x7f616c0a2410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6180101960 tx=0x7f617c009260 comp rx=0 tx=0).stop 2023-08-25T04:44:11.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 msgr2=0x7f6180072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f617000e9c0 tx=0x7f617000ecd0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 shutdown_connections 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f616c09ff50 0x7f616c0a2410 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61801024a0 0x7f6180073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6180100910 0x7f61800746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 --2- 172.21.15.145:0/4271679321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6180101b00 0x7f6180072d10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 >> 172.21.15.145:0/4271679321 conn(0x7f61800fbe60 msgr2=0x7f61800fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:11.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 shutdown_connections 2023-08-25T04:44:11.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.106+0000 7f61887ff700 1 -- 172.21.15.145:0/4271679321 wait complete. 2023-08-25T04:44:11.126 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:11.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:10 smithi162 ceph-mon[130941]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/837820169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3155812482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:10 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4046953798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:10 smithi183 ceph-mon[131414]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:11.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/837820169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3155812482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:10 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4046953798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:11.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 -- 172.21.15.145:0/1793624931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f8101b20 msgr2=0x7f81f8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 --2- 172.21.15.145:0/1793624931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f8101b20 0x7f81f8101f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81e80099c0 tx=0x7f81e8009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 -- 172.21.15.145:0/1793624931 shutdown_connections 2023-08-25T04:44:11.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 --2- 172.21.15.145:0/1793624931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f81024c0 0x7f81f8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 --2- 172.21.15.145:0/1793624931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8100930 0x7f81f8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 --2- 172.21.15.145:0/1793624931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f8101b20 0x7f81f8101f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 -- 172.21.15.145:0/1793624931 >> 172.21.15.145:0/1793624931 conn(0x7f81f80fbe60 msgr2=0x7f81f80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:11.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 -- 172.21.15.145:0/1793624931 shutdown_connections 2023-08-25T04:44:11.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.322+0000 7f81ffdff700 1 -- 172.21.15.145:0/1793624931 wait complete. 2023-08-25T04:44:11.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 Processor -- start 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 -- start start 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8101b20 0x7f81f8198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81024c0 0x7f81f819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a0320 con 0x7f81f8100930 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a0460 con 0x7f81f81024c0 2023-08-25T04:44:11.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.323+0000 7f81ffdff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81a05f0 con 0x7f81f8101b20 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42970/0 (socket says 172.21.15.145:42970) 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 -- 172.21.15.145:0/2950832797 learned_addr learned my addr 172.21.15.145:0/2950832797 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fe39c700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81024c0 0x7f81f819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fd39a700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8101b20 0x7f81f8198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 -- 172.21.15.145:0/2950832797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8101b20 msgr2=0x7f81f8198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8101b20 0x7f81f8198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 -- 172.21.15.145:0/2950832797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81024c0 msgr2=0x7f81f819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81024c0 0x7f81f819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 -- 172.21.15.145:0/2950832797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80095f0 con 0x7f81f8100930 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81fdb9b700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e970 tx=0x7f81f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4009790 con 0x7f81f8100930 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.324+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81f400c7e0 con 0x7f81f8100930 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.325+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f81a08d0 con 0x7f81f8100930 2023-08-25T04:44:11.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.325+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003c70 con 0x7f81f8100930 2023-08-25T04:44:11.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.325+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f81a0e30 con 0x7f81f8100930 2023-08-25T04:44:11.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.326+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f8192f30 con 0x7f81f8100930 2023-08-25T04:44:11.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.327+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f81f400cc50 con 0x7f81f8100930 2023-08-25T04:44:11.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.329+0000 7f81eeffd700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 0x7f81e40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.329+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f81f4014070 con 0x7f81f8100930 2023-08-25T04:44:11.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.329+0000 7f81fd39a700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 0x7f81e40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.329+0000 7f81fd39a700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 0x7f81e40a24e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81e800b580 tx=0x7f81e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:11.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.332+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f81f40950f0 con 0x7f81f8100930 2023-08-25T04:44:11.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.575+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81f802cc20 con 0x7f81f8100930 2023-08-25T04:44:11.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.576+0000 7f81eeffd700 1 -- 172.21.15.145:0/2950832797 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f81f4003dd0 con 0x7f81f8100930 2023-08-25T04:44:11.577 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:11.577 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:11.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.578+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 msgr2=0x7f81e40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.578+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 0x7f81e40a24e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81e800b580 tx=0x7f81e8005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.578+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 msgr2=0x7f81f81987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.578+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e970 tx=0x7f81f400ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:11.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 shutdown_connections 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f81e40a0020 0x7f81e40a24e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81024c0 0x7f81f819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f8100930 0x7f81f81987a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 --2- 172.21.15.145:0/2950832797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81f8101b20 0x7f81f8198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 >> 172.21.15.145:0/2950832797 conn(0x7f81f80fbe60 msgr2=0x7f81f8107400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 shutdown_connections 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.579+0000 7f81ffdff700 1 -- 172.21.15.145:0/2950832797 wait complete. 2023-08-25T04:44:11.581 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:11.598 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 -- 172.21.15.145:0/703464772 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc101b10 msgr2=0x7f34fc101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 --2- 172.21.15.145:0/703464772 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc101b10 0x7f34fc101f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0099c0 tx=0x7f34ec009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 -- 172.21.15.145:0/703464772 shutdown_connections 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 --2- 172.21.15.145:0/703464772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34fc1024b0 0x7f34fc108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 --2- 172.21.15.145:0/703464772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc100920 0x7f34fc100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 --2- 172.21.15.145:0/703464772 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc101b10 0x7f34fc101f70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 -- 172.21.15.145:0/703464772 >> 172.21.15.145:0/703464772 conn(0x7f34fc0fbe50 msgr2=0x7f34fc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 -- 172.21.15.145:0/703464772 shutdown_connections 2023-08-25T04:44:11.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.799+0000 7f35045ff700 1 -- 172.21.15.145:0/703464772 wait complete. 2023-08-25T04:44:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.800+0000 7f35045ff700 1 Processor -- start 2023-08-25T04:44:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.800+0000 7f35045ff700 1 -- start start 2023-08-25T04:44:11.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.800+0000 7f35045ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34fc100920 0x7f34fc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.800+0000 7f35045ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 0x7f34fc198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f35045ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f35045ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a03e0 con 0x7f34fc1024b0 2023-08-25T04:44:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f35045ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a0520 con 0x7f34fc101b10 2023-08-25T04:44:11.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f35045ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a06b0 con 0x7f34fc100920 2023-08-25T04:44:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42980/0 (socket says 172.21.15.145:42980) 2023-08-25T04:44:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 -- 172.21.15.145:0/2425984399 learned_addr learned my addr 172.21.15.145:0/2425984399 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3501b9a700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 0x7f34fc198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 -- 172.21.15.145:0/2425984399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34fc100920 msgr2=0x7f34fc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34fc100920 0x7f34fc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 -- 172.21.15.145:0/2425984399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 msgr2=0x7f34fc198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:11.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 0x7f34fc198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:11.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 -- 172.21.15.145:0/2425984399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0095f0 con 0x7f34fc1024b0 2023-08-25T04:44:11.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3501b9a700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 0x7f34fc198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.801+0000 7f3502b9c700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ef30 tx=0x7f34f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.802+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4018070 con 0x7f34fc1024b0 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.802+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc1a0930 con 0x7f34fc1024b0 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.802+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f400f040 con 0x7f34fc1024b0 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.802+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400cdb0 con 0x7f34fc1024b0 2023-08-25T04:44:11.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.802+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1a0e40 con 0x7f34fc1024b0 2023-08-25T04:44:11.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.803+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc192f70 con 0x7f34fc1024b0 2023-08-25T04:44:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.805+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f34f4003a00 con 0x7f34fc1024b0 2023-08-25T04:44:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.806+0000 7f34f37fe700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 0x7f34e80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.806+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f34f40cb110 con 0x7f34fc1024b0 2023-08-25T04:44:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.807+0000 7f350239b700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 0x7f34e80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:11.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.807+0000 7f350239b700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 0x7f34e80a2460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34fc101970 tx=0x7f34f80096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:11.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:11.810+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f34f4094bd0 con 0x7f34fc1024b0 2023-08-25T04:44:12.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.015+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34fc10c180 con 0x7f34fc1024b0 2023-08-25T04:44:12.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.016+0000 7f34f37fe700 1 -- 172.21.15.145:0/2425984399 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f34f40981f0 con 0x7f34fc1024b0 2023-08-25T04:44:12.017 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:12.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 msgr2=0x7f34e80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 0x7f34e80a2460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34fc101970 tx=0x7f34f80096c0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 msgr2=0x7f34fc19fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ef30 tx=0x7f34f400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 shutdown_connections 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f34e809ffa0 0x7f34e80a2460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34fc1024b0 0x7f34fc19fe10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34fc100920 0x7f34fc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 --2- 172.21.15.145:0/2425984399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34fc101b10 0x7f34fc198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 >> 172.21.15.145:0/2425984399 conn(0x7f34fc0fbe50 msgr2=0x7f34fc107400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 shutdown_connections 2023-08-25T04:44:12.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.019+0000 7f35045ff700 1 -- 172.21.15.145:0/2425984399 wait complete. 2023-08-25T04:44:12.036 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:11 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4271679321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:12.037 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:11 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2950832797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:12.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:11 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4271679321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:11 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2950832797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:11 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4271679321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:11 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2950832797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:12.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 -- 172.21.15.145:0/612087166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 msgr2=0x7fd958100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/612087166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd958100d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd954009920 tx=0x7fd954009c30 comp rx=0 tx=0).stop 2023-08-25T04:44:12.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 -- 172.21.15.145:0/612087166 shutdown_connections 2023-08-25T04:44:12.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/612087166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd958108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/612087166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd958100d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/612087166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 0x7fd958101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 -- 172.21.15.145:0/612087166 >> 172.21.15.145:0/612087166 conn(0x7fd9580fbe90 msgr2=0x7fd9580fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 -- 172.21.15.145:0/612087166 shutdown_connections 2023-08-25T04:44:12.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.234+0000 7fd95f9ff700 1 -- 172.21.15.145:0/612087166 wait complete. 2023-08-25T04:44:12.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.235+0000 7fd95f9ff700 1 Processor -- start 2023-08-25T04:44:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 -- start start 2023-08-25T04:44:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd9581966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 0x7fd958196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819e250 con 0x7fd9581024d0 2023-08-25T04:44:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819e390 con 0x7fd958100940 2023-08-25T04:44:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95f9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95819e520 con 0x7fd958101b30 2023-08-25T04:44:12.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95df9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95df9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43006/0 (socket says 172.21.15.145:43006) 2023-08-25T04:44:12.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95d79b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd9581966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95df9c700 1 -- 172.21.15.145:0/1547952535 learned_addr learned my addr 172.21.15.145:0/1547952535 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.236+0000 7fd95cf9a700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 0x7fd958196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 -- 172.21.15.145:0/1547952535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 msgr2=0x7fd958196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 0x7fd958196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 -- 172.21.15.145:0/1547952535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 msgr2=0x7fd9581966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd9581966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 -- 172.21.15.145:0/1547952535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948009520 con 0x7fd9581024d0 2023-08-25T04:44:12.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95df9c700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e3c0 tx=0x7fd95000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500090d0 con 0x7fd9581024d0 2023-08-25T04:44:12.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd950009d00 con 0x7fd9581024d0 2023-08-25T04:44:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95000f040 con 0x7fd9581024d0 2023-08-25T04:44:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950007c20 con 0x7fd9581024d0 2023-08-25T04:44:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.237+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954009520 con 0x7fd9581024d0 2023-08-25T04:44:12.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.238+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd958107f90 con 0x7fd9581024d0 2023-08-25T04:44:12.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.240+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd950003a00 con 0x7fd9581024d0 2023-08-25T04:44:12.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.241+0000 7fd94e7fc700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 0x7fd9440a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.241+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd9500cb330 con 0x7fd9581024d0 2023-08-25T04:44:12.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.242+0000 7fd95d79b700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 0x7fd9440a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.242+0000 7fd95d79b700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 0x7fd9440a2460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098f0 tx=0x7fd954019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.245+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd950094df0 con 0x7fd9581024d0 2023-08-25T04:44:12.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.488+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd95810c1a0 con 0x7fd9581024d0 2023-08-25T04:44:12.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.488+0000 7fd94e7fc700 1 -- 172.21.15.145:0/1547952535 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7fd950098410 con 0x7fd9581024d0 2023-08-25T04:44:12.490 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:12.490 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:12.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 msgr2=0x7fd9440a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 0x7fd9440a2460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098f0 tx=0x7fd954019040 comp rx=0 tx=0).stop 2023-08-25T04:44:12.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 msgr2=0x7fd95819dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd95000e3c0 tx=0x7fd95000e6d0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 shutdown_connections 2023-08-25T04:44:12.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd94409ffa0 0x7fd9440a2460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9581024d0 0x7fd95819dc80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958100940 0x7fd9581966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 --2- 172.21.15.145:0/1547952535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd958101b30 0x7fd958196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 >> 172.21.15.145:0/1547952535 conn(0x7fd9580fbe90 msgr2=0x7fd9581050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 shutdown_connections 2023-08-25T04:44:12.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.491+0000 7fd95f9ff700 1 -- 172.21.15.145:0/1547952535 wait complete. 2023-08-25T04:44:12.495 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:12.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-25T04:42:49.862390+0000', 'modified': '2023-08-25T04:43:16.556417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24562, 'mds_1': 24557, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/557521219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 557521219}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 557521219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/4241662550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 4241662550}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 4241662550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6837/1048214793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1048214793}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1048214793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.183:6839/3401840698', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3401840698}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3401840698}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.162:6839/2860621511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 2860621511}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 2860621511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.145:6841/3986940142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6840', 'nonce': 3986940142}, {'type': 'v1', 'addr': '172.21.15.145:6841', 'nonce': 3986940142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T04:44:12.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-08-25T04:44:12.512 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-25T04:44:12.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/301843645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 msgr2=0x7f9d54072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/301843645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54072d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009990 tx=0x7f9d44009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/301843645 shutdown_connections 2023-08-25T04:44:12.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/301843645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54108810 0x7f9d5410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/301843645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54072d50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/301843645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54073320 0x7f9d540737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/301843645 >> 172.21.15.145:0/301843645 conn(0x7f9d540fbed0 msgr2=0x7f9d540fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/301843645 shutdown_connections 2023-08-25T04:44:12.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.710+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/301843645 wait complete. 2023-08-25T04:44:12.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.711+0000 7f9d5a3ff700 1 Processor -- start 2023-08-25T04:44:12.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.711+0000 7f9d5a3ff700 1 -- start start 2023-08-25T04:44:12.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54108810 0x7f9d5419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541a03f0 con 0x7f9d54074900 2023-08-25T04:44:12.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541a0530 con 0x7f9d54073320 2023-08-25T04:44:12.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5a3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541a06c0 con 0x7f9d54108810 2023-08-25T04:44:12.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60218/0 (socket says 172.21.15.145:60218) 2023-08-25T04:44:12.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d53fff700 1 -- 172.21.15.145:0/2578233463 learned_addr learned my addr 172.21.15.145:0/2578233463 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d5899c700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54108810 0x7f9d5419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 -- 172.21.15.145:0/2578233463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54108810 msgr2=0x7f9d5419fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54108810 0x7f9d5419fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 -- 172.21.15.145:0/2578233463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 msgr2=0x7f9d54198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d4bfff700 1 -- 172.21.15.145:0/2578233463 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d440095f0 con 0x7f9d54074900 2023-08-25T04:44:12.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.712+0000 7f9d53fff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:12.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54198db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00e9f0 tx=0x7f9d3c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d51ffb700 1 -- 172.21.15.145:0/2578233463 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00c9c0 con 0x7f9d54074900 2023-08-25T04:44:12.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d541a09a0 con 0x7f9d54074900 2023-08-25T04:44:12.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d51ffb700 1 -- 172.21.15.145:0/2578233463 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3c010430 con 0x7f9d54074900 2023-08-25T04:44:12.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d51ffb700 1 -- 172.21.15.145:0/2578233463 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c018680 con 0x7f9d54074900 2023-08-25T04:44:12.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.713+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541a0f00 con 0x7f9d54074900 2023-08-25T04:44:12.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.716+0000 7f9d51ffb700 1 -- 172.21.15.145:0/2578233463 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9d3c010860 con 0x7f9d54074900 2023-08-25T04:44:12.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.717+0000 7f9d6ddedb80 1 Processor -- start 2023-08-25T04:44:12.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 -- start start 2023-08-25T04:44:12.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d51ffb700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 0x7f9d400a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d51ffb700 1 -- 172.21.15.145:0/2578233463 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d3c014070 con 0x7f9d54074900 2023-08-25T04:44:12.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e83b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e83c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e7a2760 con 0x55e47e83b2e0 2023-08-25T04:44:12.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e793730 con 0x55e47e83a980 2023-08-25T04:44:12.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e791050 con 0x55e47e83bd00 2023-08-25T04:44:12.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d5899c700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 0x7f9d400a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e83b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60228/0 (socket says 172.21.15.145:60228) 2023-08-25T04:44:12.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d53fff700 1 -- 172.21.15.145:0/2887111124 learned_addr learned my addr 172.21.15.145:0/2887111124 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.718+0000 7f9d5899c700 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e83c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 -- 172.21.15.145:0/2887111124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 msgr2=0x55e47e83c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e83c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 -- 172.21.15.145:0/2887111124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 msgr2=0x55e47e83b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e83b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 -- 172.21.15.145:0/2887111124 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e47e60c5e0 con 0x55e47e83a980 2023-08-25T04:44:12.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e83b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:12.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d5899c700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 0x7f9d400a2410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009ec0 tx=0x7f9d4c00fc40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d53fff700 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d44005f30 tx=0x7f9d44005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5799e9d8152880e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d4a7fc700 1 -- 172.21.15.145:0/2887111124 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4401d070 con 0x55e47e83a980 2023-08-25T04:44:12.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d4a7fc700 1 -- 172.21.15.145:0/2887111124 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4401e040 con 0x55e47e83a980 2023-08-25T04:44:12.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.719+0000 7f9d4a7fc700 1 -- 172.21.15.145:0/2887111124 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004e30 con 0x55e47e83a980 2023-08-25T04:44:12.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/2887111124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 msgr2=0x55e47e83ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d44005f30 tx=0x7f9d44005c00 comp rx=0 tx=0).stop 2023-08-25T04:44:12.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/2887111124 shutdown_connections 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e83c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83a980 0x55e47e83ada0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/2887111124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e83b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/2887111124 >> 172.21.15.145:0/2887111124 conn(0x55e47e7af090 msgr2=0x55e47e835250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/2887111124 shutdown_connections 2023-08-25T04:44:12.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.720+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/2887111124 wait complete. 2023-08-25T04:44:12.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 -- start start 2023-08-25T04:44:12.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 Processor -- start 2023-08-25T04:44:12.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e844810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83bd00 0x55e47e84b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e78c940 con 0x55e47e83bd00 2023-08-25T04:44:12.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e72ab60 con 0x55e47e83b2e0 2023-08-25T04:44:12.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e72a860 con 0x55e47e83a980 2023-08-25T04:44:12.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d6ddedb80 10 client.0 _ll_register_callbacks cb 0x55e47e674a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:12.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.721+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d5899c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83bd00 0x55e47e84b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43638/0 (socket says 172.21.15.145:43638) 2023-08-25T04:44:12.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 -- 172.21.15.145:0/3230575144 learned_addr learned my addr 172.21.15.145:0/3230575144 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e844810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 -- 172.21.15.145:0/3230575144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 msgr2=0x55e47e844810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e844810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 -- 172.21.15.145:0/3230575144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83bd00 msgr2=0x55e47e84b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83bd00 0x55e47e84b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 -- 172.21.15.145:0/3230575144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541a0f00 con 0x55e47e83a980 2023-08-25T04:44:12.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ce0 tx=0x7f9d44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87c161ad330ff3b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d337fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:44:12.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.722+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4401d040 con 0x55e47e83a980 2023-08-25T04:44:12.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e47e80a170 con 0x55e47e83a980 2023-08-25T04:44:12.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d440055d0 con 0x55e47e83a980 2023-08-25T04:44:12.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4400f460 con 0x55e47e83a980 2023-08-25T04:44:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 10 client.25102 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f9d5410b8a0 con 0x55e47e83a980 2023-08-25T04:44:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e47e80a170 con 0x55e47e83a980 2023-08-25T04:44:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d44022540 con 0x55e47e83a980 2023-08-25T04:44:12.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 10 client.25102 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:12.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 10 client.25102 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T04:44:12.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 20 client.25102 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:12.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e84fa20 0x55e47e84e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 4 client.25102 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:12.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.723+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e47e732b50 con 0x55e47e84fa20 2023-08-25T04:44:12.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.724+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e84fa20 0x55e47e84e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.724+0000 7f9d337fe700 10 client.25102 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:12.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.724+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e84fa20 0x55e47e84e780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f2cbee0489d6635 server_cookie=6d8620494b4caa56 in_seq=0 out_seq=0 2023-08-25T04:44:12.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d337fe700 1 -- 172.21.15.145:0/3230575144 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e47e732b50 con 0x55e47e84fa20 2023-08-25T04:44:12.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d337fe700 10 client.25102 handle_command_reply: tid=0 2023-08-25T04:44:12.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 client.25102 shutdown 2023-08-25T04:44:12.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 msgr2=0x55e47e8442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ce0 tx=0x7f9d44004730 comp rx=0 tx=0).stop 2023-08-25T04:44:12.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 shutdown_connections 2023-08-25T04:44:12.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e84fa20 0x55e47e84e780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83bd00 0x55e47e84b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d337fe700 0 client.25102 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:12.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e8442d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3230575144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e844810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 >> 172.21.15.145:0/3230575144 conn(0x55e47e7af090 msgr2=0x55e47e82faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 shutdown_connections 2023-08-25T04:44:12.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.725+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3230575144 wait complete. 2023-08-25T04:44:12.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.726+0000 7f9d6ddedb80 20 client.25102 trim_cache size 0 max 16384 2023-08-25T04:44:12.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.743+0000 7f9d6ddedb80 1 Processor -- start 2023-08-25T04:44:12.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.743+0000 7f9d6ddedb80 1 -- start start 2023-08-25T04:44:12.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e84ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e84b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e60e8d0 con 0x55e47e83a980 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e734470 con 0x55e47e83b2e0 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e61ade0 con 0x55e47e83bd00 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43048/0 (socket says 172.21.15.145:43048) 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 -- 172.21.15.145:0/99160531 learned_addr learned my addr 172.21.15.145:0/99160531 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e84ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d5899c700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e84b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 -- 172.21.15.145:0/99160531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 msgr2=0x55e47e84b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e84b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 -- 172.21.15.145:0/99160531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 msgr2=0x55e47e84ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e84ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d53fff700 1 -- 172.21.15.145:0/99160531 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e47e5b8d90 con 0x55e47e83a980 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.744+0000 7f9d5899c700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e84b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d53fff700 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dd84ff8b1a9f4b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d4affd700 1 -- 172.21.15.145:0/99160531 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400bee0 con 0x55e47e83a980 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d4affd700 1 -- 172.21.15.145:0/99160531 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d44005910 con 0x55e47e83a980 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d4affd700 1 -- 172.21.15.145:0/99160531 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400f920 con 0x55e47e83a980 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/99160531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 msgr2=0x55e47e84e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44005bc0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/99160531 shutdown_connections 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83bd00 0x55e47e84b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83a980 0x55e47e84e780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/99160531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83b2e0 0x55e47e84ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.745+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/99160531 >> 172.21.15.145:0/99160531 conn(0x55e47e7af090 msgr2=0x55e47e82ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.746+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/99160531 shutdown_connections 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.746+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/99160531 wait complete. 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.746+0000 7f9d6ddedb80 1 -- start start 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.746+0000 7f9d6ddedb80 1 Processor -- start 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e82fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 0x55e47e84ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e84b460 con 0x55e47e83b2e0 2023-08-25T04:44:12.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e84b5d0 con 0x55e47e83bd00 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e47e84b790 con 0x55e47e83a980 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d6ddedb80 10 client.0 _ll_register_callbacks cb 0x7f9d3c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43050/0 (socket says 172.21.15.145:43050) 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 -- 172.21.15.145:0/3144076999 learned_addr learned my addr 172.21.15.145:0/3144076999 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e82fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d5899c700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 0x55e47e84ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 -- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 msgr2=0x55e47e82fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e82fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 -- 172.21.15.145:0/3144076999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 msgr2=0x55e47e84ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 0x55e47e84ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d4bfff700 1 -- 172.21.15.145:0/3144076999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5410b8a0 con 0x55e47e83b2e0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.747+0000 7f9d5899c700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 0x55e47e84ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d4bfff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c095280 tx=0x7f9d3c0b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed89872f09f1e660 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0a0b90 con 0x55e47e83b2e0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e47e82d860 con 0x55e47e83b2e0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d3c0105a0 con 0x55e47e83b2e0 2023-08-25T04:44:12.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d3c0cb370 con 0x55e47e83b2e0 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 10 client.15096 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e47e5a1c10 con 0x55e47e83b2e0 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e47e82d860 con 0x55e47e83b2e0 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d3c099070 con 0x55e47e83b2e0 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 20 client.15096 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:12.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e8a7ff0 0x55e47e8a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 4 client.15096 mds_command: new command op to 24532 tid=0[{"prefix": "status"}] 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.748+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e47e785920 con 0x55e47e8a7ff0 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.749+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e8a7ff0 0x55e47e8a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.749+0000 7f9d337fe700 10 client.15096 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.749+0000 7f9d53fff700 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e8a7ff0 0x55e47e8a4350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5becd2a8c242c4be server_cookie=1d7a324c8d105126 in_seq=0 out_seq=0 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d337fe700 1 -- 172.21.15.145:0/3144076999 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e47e785920 con 0x55e47e8a7ff0 2023-08-25T04:44:12.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d337fe700 10 client.15096 handle_command_reply: tid=0 2023-08-25T04:44:12.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 client.15096 shutdown 2023-08-25T04:44:12.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 msgr2=0x55e47e84a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c095280 tx=0x7f9d3c0b8040 comp rx=0 tx=0).stop 2023-08-25T04:44:12.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 shutdown_connections 2023-08-25T04:44:12.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55e47e8a7ff0 0x55e47e8a4350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e47e83bd00 0x55e47e84ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d337fe700 0 client.15096 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:12.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47e83a980 0x55e47e82fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 --2- 172.21.15.145:0/3144076999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e47e83b2e0 0x55e47e84a7b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 >> 172.21.15.145:0/3144076999 conn(0x55e47e7af090 msgr2=0x55e47e84e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 shutdown_connections 2023-08-25T04:44:12.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 1 -- 172.21.15.145:0/3144076999 wait complete. 2023-08-25T04:44:12.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.750+0000 7f9d6ddedb80 20 client.15096 trim_cache size 0 max 16384 2023-08-25T04:44:12.783 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:12.783 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:12.783 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 2, 2023-08-25T04:44:12.783 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24532, 2023-08-25T04:44:12.783 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 57.196044031, 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:12.784 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 98.225430571000004 2023-08-25T04:44:12.785 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:12.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 msgr2=0x7f9d400a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 0x7f9d400a2410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009ec0 tx=0x7f9d4c00fc40 comp rx=0 tx=0).stop 2023-08-25T04:44:12.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 msgr2=0x7f9d54198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54198db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00e9f0 tx=0x7f9d3c00ed00 comp rx=0 tx=0).stop 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 shutdown_connections 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9d4009ff50 0x7f9d400a2410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d54108810 0x7f9d5419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d54073320 0x7f9d54198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 --2- 172.21.15.145:0/2578233463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d54074900 0x7f9d54198db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 >> 172.21.15.145:0/2578233463 conn(0x7f9d540fbed0 msgr2=0x7f9d540fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 shutdown_connections 2023-08-25T04:44:12.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.751+0000 7f9d5a3ff700 1 -- 172.21.15.145:0/2578233463 wait complete. 2023-08-25T04:44:12.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T04:44:12.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 -- 172.21.15.145:0/3378614057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf041024a0 msgr2=0x7faf04108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 --2- 172.21.15.145:0/3378614057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf041024a0 0x7faf04108d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faefc00b0d0 tx=0x7faefc00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 -- 172.21.15.145:0/3378614057 shutdown_connections 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 --2- 172.21.15.145:0/3378614057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf041024a0 0x7faf04108d50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 --2- 172.21.15.145:0/3378614057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 --2- 172.21.15.145:0/3378614057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf04101b00 0x7faf04101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 -- 172.21.15.145:0/3378614057 >> 172.21.15.145:0/3378614057 conn(0x7faf040fbe60 msgr2=0x7faf040fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.987+0000 7faf08dff700 1 -- 172.21.15.145:0/3378614057 shutdown_connections 2023-08-25T04:44:12.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.988+0000 7faf08dff700 1 -- 172.21.15.145:0/3378614057 wait complete. 2023-08-25T04:44:12.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 Processor -- start 2023-08-25T04:44:12.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 -- start start 2023-08-25T04:44:12.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 0x7faf04198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 0x7faf0419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041a04b0 con 0x7faf04101b00 2023-08-25T04:44:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041a05f0 con 0x7faf04100910 2023-08-25T04:44:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf08dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041a0780 con 0x7faf041024a0 2023-08-25T04:44:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faefbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 0x7faf04198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf0259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf0259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:34684/0 (socket says 172.21.15.145:34684) 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 0x7faf0419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.989+0000 7faf0259c700 1 -- 172.21.15.145:0/4092106197 learned_addr learned my addr 172.21.15.145:0/4092106197 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 -- 172.21.15.145:0/4092106197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 msgr2=0x7faf04198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 -- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 msgr2=0x7faf04198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 0x7faf04198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 -- 172.21.15.145:0/4092106197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009520 con 0x7faf041024a0 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faefbfff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 0x7faf04198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf02d9d700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 0x7faf0419fee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faefc007a30 tx=0x7faefc008eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faefb7fe700 1 -- 172.21.15.145:0/4092106197 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00e040 con 0x7faf041024a0 2023-08-25T04:44:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc009d00 con 0x7faf041024a0 2023-08-25T04:44:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faefb7fe700 1 -- 172.21.15.145:0/4092106197 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc00bd30 con 0x7faf041024a0 2023-08-25T04:44:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faefb7fe700 1 -- 172.21.15.145:0/4092106197 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc01d470 con 0x7faf041024a0 2023-08-25T04:44:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.990+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf041a0cf0 con 0x7faf041024a0 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.993+0000 7faefb7fe700 1 -- 172.21.15.145:0/4092106197 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7faefc005020 con 0x7faf041024a0 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 Processor -- start 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faefb7fe700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 0x7faef00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faefb7fe700 1 -- 172.21.15.145:0/4092106197 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faefc0cc4b0 con 0x7faf041024a0 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 -- start start 2023-08-25T04:44:13.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf0259c700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 0x7faef00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a564720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a565180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a4cb760 con 0x55b89a564d00 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a4bc730 con 0x55b89a563980 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.995+0000 7faf1c720b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a4ba050 con 0x55b89a5642e0 2023-08-25T04:44:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:34690/0 (socket says 172.21.15.145:34690) 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 -- 172.21.15.145:0/3352452865 learned_addr learned my addr 172.21.15.145:0/3352452865 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf02d9d700 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a565180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faefbfff700 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a564720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 0x7faef00a2410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faef4005950 tx=0x7faef4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 -- 172.21.15.145:0/3352452865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 msgr2=0x55b89a564720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a564720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 -- 172.21.15.145:0/3352452865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 msgr2=0x55b89a565180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a565180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 -- 172.21.15.145:0/3352452865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89a3355e0 con 0x55b89a563980 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.996+0000 7faf0259c700 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faef4012fd0 tx=0x7faef4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a36ad900eca4d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faef8ff9700 1 -- 172.21.15.145:0/3352452865 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef401cd80 con 0x55b89a563980 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faef8ff9700 1 -- 172.21.15.145:0/3352452865 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faef40044e0 con 0x55b89a563980 2023-08-25T04:44:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faef8ff9700 1 -- 172.21.15.145:0/3352452865 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef401b690 con 0x55b89a563980 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 -- 172.21.15.145:0/3352452865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 msgr2=0x55b89a563da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faef4012fd0 tx=0x7faef4014000 comp rx=0 tx=0).stop 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 -- 172.21.15.145:0/3352452865 shutdown_connections 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a565180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a563da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3352452865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a564720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 -- 172.21.15.145:0/3352452865 >> 172.21.15.145:0/3352452865 conn(0x55b89a4d8090 msgr2=0x55b89a55e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 -- 172.21.15.145:0/3352452865 shutdown_connections 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.997+0000 7faf1c720b80 1 -- 172.21.15.145:0/3352452865 wait complete. 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.998+0000 7faf1c720b80 1 -- start start 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.998+0000 7faf1c720b80 1 Processor -- start 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a56d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a56d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a4b5940 con 0x55b89a564d00 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a453b60 con 0x55b89a5642e0 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a453860 con 0x55b89a563980 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf1c720b80 10 client.0 _ll_register_callbacks cb 0x55b89a39da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60642/0 (socket says 172.21.15.145:60642) 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 -- 172.21.15.145:0/2727187453 learned_addr learned my addr 172.21.15.145:0/2727187453 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faefbfff700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a56d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf0259c700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a56d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 -- 172.21.15.145:0/2727187453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 msgr2=0x55b89a56d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a56d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 -- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 msgr2=0x55b89a56d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a56d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 -- 172.21.15.145:0/2727187453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4017050 con 0x55b89a564d00 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faefc009fd0 tx=0x7faefc099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b3d7119b0e6bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:12.999+0000 7faee9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0cc2e0 con 0x55b89a564d00 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faefc011e70 con 0x55b89a564d00 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b89a533170 con 0x55b89a564d00 2023-08-25T04:44:13.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faefc099590 con 0x55b89a564d00 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 10 client.15108 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b89a2e1d90 con 0x55b89a564d00 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b89a533170 con 0x55b89a564d00 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faefc099120 con 0x55b89a564d00 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:13.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T04:44:13.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 20 client.15108 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:13.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a578c70 0x55b89a577870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 4 client.15108 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:13.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.000+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89a45bb50 con 0x55b89a578c70 2023-08-25T04:44:13.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.001+0000 7faf0259c700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a578c70 0x55b89a577870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.001+0000 7faf0259c700 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a578c70 0x55b89a577870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e21e4cc7de10023e server_cookie=9b048d4fa3a333ae in_seq=0 out_seq=0 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.001+0000 7faee9ffb700 10 client.15108 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faee9ffb700 1 -- 172.21.15.145:0/2727187453 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b89a45bb50 con 0x55b89a578c70 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faee9ffb700 10 client.15108 handle_command_reply: tid=0 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 client.15108 shutdown 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 msgr2=0x55b89a574940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faefc009fd0 tx=0x7faefc099a70 comp rx=0 tx=0).stop 2023-08-25T04:44:13.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 shutdown_connections 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a578c70 0x55b89a577870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a574940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a56d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faee9ffb700 0 client.15108 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2727187453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a56d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.002+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 >> 172.21.15.145:0/2727187453 conn(0x55b89a4d8090 msgr2=0x55b89a558af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.003+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 shutdown_connections 2023-08-25T04:44:13.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.003+0000 7faf1c720b80 1 -- 172.21.15.145:0/2727187453 wait complete. 2023-08-25T04:44:13.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.003+0000 7faf1c720b80 20 client.15108 trim_cache size 0 max 16384 2023-08-25T04:44:13.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.020+0000 7faf1c720b80 1 Processor -- start 2023-08-25T04:44:13.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.020+0000 7faf1c720b80 1 -- start start 2023-08-25T04:44:13.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a56c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a574900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a3378d0 con 0x55b89a564d00 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a475e20 con 0x55b89a563980 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf1c720b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a471f00 con 0x55b89a5642e0 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60644/0 (socket says 172.21.15.145:60644) 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 -- 172.21.15.145:0/2336238329 learned_addr learned my addr 172.21.15.145:0/2336238329 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faefbfff700 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a574900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf0259c700 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a56c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 -- 172.21.15.145:0/2336238329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 msgr2=0x55b89a574900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a574900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 -- 172.21.15.145:0/2336238329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 msgr2=0x55b89a56c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a56c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.021+0000 7faf02d9d700 1 -- 172.21.15.145:0/2336238329 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89a2cac10 con 0x55b89a564d00 2023-08-25T04:44:13.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faf02d9d700 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faefc096130 tx=0x7faefc00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10eeb54d0bf4861 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faef97fa700 1 -- 172.21.15.145:0/2336238329 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc09a330 con 0x55b89a564d00 2023-08-25T04:44:13.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faef97fa700 1 -- 172.21.15.145:0/2336238329 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc099590 con 0x55b89a564d00 2023-08-25T04:44:13.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faef97fa700 1 -- 172.21.15.145:0/2336238329 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc099cc0 con 0x55b89a564d00 2023-08-25T04:44:13.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faf1c720b80 1 -- 172.21.15.145:0/2336238329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 msgr2=0x55b89a567bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.022+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faefc096130 tx=0x7faefc00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 -- 172.21.15.145:0/2336238329 shutdown_connections 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a567bc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a563980 0x55b89a56c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 --2- 172.21.15.145:0/2336238329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a5642e0 0x55b89a574900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 -- 172.21.15.145:0/2336238329 >> 172.21.15.145:0/2336238329 conn(0x55b89a4d8090 msgr2=0x55b89a557ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 -- 172.21.15.145:0/2336238329 shutdown_connections 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 -- 172.21.15.145:0/2336238329 wait complete. 2023-08-25T04:44:13.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 -- start start 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.023+0000 7faf1c720b80 1 Processor -- start 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a558c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a573f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a574680 con 0x55b89a564d00 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a5747f0 con 0x55b89a5642e0 2023-08-25T04:44:13.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89a5749b0 con 0x55b89a563980 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf1c720b80 10 client.0 _ll_register_callbacks cb 0x7faefc009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a573f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:34710/0 (socket says 172.21.15.145:34710) 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 -- 172.21.15.145:0/3859501423 learned_addr learned my addr 172.21.15.145:0/3859501423 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf0259c700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a558c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 -- 172.21.15.145:0/3859501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 msgr2=0x55b89a558c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a558c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 -- 172.21.15.145:0/3859501423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 msgr2=0x55b89a573f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a573f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faefbfff700 1 -- 172.21.15.145:0/3859501423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89a2e1d90 con 0x55b89a5642e0 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.024+0000 7faf02d9d700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a573f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faefbfff700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faeec012d30 tx=0x7faeec00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bff1bb782ee3e18 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faee9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:44:13.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec017040 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b89a556860 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faeec00fd20 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faeec00fee0 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faf1c720b80 10 client.25115 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.025+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b89a308200 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b89a556860 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faeec010430 con 0x55b89a5642e0 2023-08-25T04:44:13.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 10 client.25115 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:13.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 10 client.25115 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T04:44:13.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 20 client.25115 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:13.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a5d14a0 0x55b89a5cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 4 client.25115 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-08-25T04:44:13.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b89a4ae920 con 0x55b89a5d14a0 2023-08-25T04:44:13.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.026+0000 7faf0259c700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a5d14a0 0x55b89a5cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf0259c700 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a5d14a0 0x55b89a5cd920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c38b1422b19a9eb server_cookie=c1a24197807ad367 in_seq=0 out_seq=0 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faee9ffb700 10 client.25115 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faee9ffb700 1 -- 172.21.15.145:0/3859501423 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b89a4ae920 con 0x55b89a5d14a0 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faee9ffb700 10 client.25115 handle_command_reply: tid=0 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 client.25115 shutdown 2023-08-25T04:44:13.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 msgr2=0x55b89a573940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faeec012d30 tx=0x7faeec00f6a0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 shutdown_connections 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b89a5d14a0 0x55b89a5cd920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b89a564d00 0x55b89a573f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b89a563980 0x55b89a558c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 --2- 172.21.15.145:0/3859501423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b89a5642e0 0x55b89a573940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faee9ffb700 0 client.25115 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:13.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 >> 172.21.15.145:0/3859501423 conn(0x55b89a4d8090 msgr2=0x55b89a56b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 shutdown_connections 2023-08-25T04:44:13.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.027+0000 7faf1c720b80 1 -- 172.21.15.145:0/3859501423 wait complete. 2023-08-25T04:44:13.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf1c720b80 20 client.25115 trim_cache size 0 max 16384 2023-08-25T04:44:13.052 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:13.052 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:13.052 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 1, 2023-08-25T04:44:13.052 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24557, 2023-08-25T04:44:13.052 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 58.486065770000003, 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 90.182662178000001 2023-08-25T04:44:13.053 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:13.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 msgr2=0x7faef00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 0x7faef00a2410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faef4005950 tx=0x7faef4009260 comp rx=0 tx=0).stop 2023-08-25T04:44:13.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 msgr2=0x7faf0419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 0x7faf0419fee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faefc007a30 tx=0x7faefc008eb0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 shutdown_connections 2023-08-25T04:44:13.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faef009ff50 0x7faef00a2410 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041024a0 0x7faf0419fee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf04100910 0x7faf04198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 --2- 172.21.15.145:0/4092106197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf04101b00 0x7faf04198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 >> 172.21.15.145:0/4092106197 conn(0x7faf040fbe60 msgr2=0x7faf041073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 shutdown_connections 2023-08-25T04:44:13.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.028+0000 7faf08dff700 1 -- 172.21.15.145:0/4092106197 wait complete. 2023-08-25T04:44:13.057 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:12 smithi145 ceph-mon[97681]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:13.057 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:12 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2425984399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.058 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:12 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1547952535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T04:44:13.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:12 smithi162 ceph-mon[130941]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:12 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2425984399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:12 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1547952535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:12 smithi183 ceph-mon[131414]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:13.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:12 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2425984399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:12 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1547952535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:13.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 -- 172.21.15.145:0/3264098297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8073380 msgr2=0x7fccf8073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 --2- 172.21.15.145:0/3264098297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8073380 0x7fccf8073800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009990 tx=0x7fcce0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 -- 172.21.15.145:0/3264098297 shutdown_connections 2023-08-25T04:44:13.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 --2- 172.21.15.145:0/3264098297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf81087f0 0x7fccf810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 --2- 172.21.15.145:0/3264098297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 0x7fccf8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 --2- 172.21.15.145:0/3264098297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf8073380 0x7fccf8073800 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 -- 172.21.15.145:0/3264098297 >> 172.21.15.145:0/3264098297 conn(0x7fccf80fbea0 msgr2=0x7fccf80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 -- 172.21.15.145:0/3264098297 shutdown_connections 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.255+0000 7fccfdbff700 1 -- 172.21.15.145:0/3264098297 wait complete. 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 Processor -- start 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 -- start start 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf8073380 0x7fccf8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 0x7fccf8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81a03d0 con 0x7fccf81087f0 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81a0510 con 0x7fccf8074960 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.256+0000 7fccfdbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81a06a0 con 0x7fccf8073380 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60674/0 (socket says 172.21.15.145:60674) 2023-08-25T04:44:13.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 -- 172.21.15.145:0/1867738721 learned_addr learned my addr 172.21.15.145:0/1867738721 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf8073380 0x7fccf8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fcceffff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 0x7fccf8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 -- 172.21.15.145:0/1867738721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf8073380 msgr2=0x7fccf8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf8073380 0x7fccf8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 -- 172.21.15.145:0/1867738721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 msgr2=0x7fccf8198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 0x7fccf8198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 -- 172.21.15.145:0/1867738721 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00095f0 con 0x7fccf81087f0 2023-08-25T04:44:13.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.257+0000 7fccf7fff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef30 tx=0x7fccf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.258+0000 7fccf57fa700 1 -- 172.21.15.145:0/1867738721 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0018070 con 0x7fccf81087f0 2023-08-25T04:44:13.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.258+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf81a0920 con 0x7fccf81087f0 2023-08-25T04:44:13.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.258+0000 7fccf57fa700 1 -- 172.21.15.145:0/1867738721 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf000f040 con 0x7fccf81087f0 2023-08-25T04:44:13.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.258+0000 7fccf57fa700 1 -- 172.21.15.145:0/1867738721 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000cd00 con 0x7fccf81087f0 2023-08-25T04:44:13.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.258+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81a0de0 con 0x7fccf81087f0 2023-08-25T04:44:13.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.262+0000 7fccf57fa700 1 -- 172.21.15.145:0/1867738721 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fccf0003a00 con 0x7fccf81087f0 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.263+0000 7fcd11432b80 1 Processor -- start 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.263+0000 7fcd11432b80 1 -- start start 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c30051eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051fd00 0x55c300520180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300486760 con 0x55c30051fd00 2023-08-25T04:44:13.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300477730 con 0x55c30051f2e0 2023-08-25T04:44:13.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcd11432b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300475050 con 0x55c30051e980 2023-08-25T04:44:13.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf57fa700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 0x7fcce40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcceffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051fd00 0x55c300520180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:34736/0 (socket says 172.21.15.145:34736) 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 -- 172.21.15.145:0/1890631890 learned_addr learned my addr 172.21.15.145:0/1890631890 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf7fff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 0x7fcce40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fcceffff700 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c30051eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf57fa700 1 -- 172.21.15.145:0/1867738721 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fccf00cb0f0 con 0x7fccf81087f0 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 -- 172.21.15.145:0/1890631890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 msgr2=0x55c30051eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c30051eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 -- 172.21.15.145:0/1890631890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051fd00 msgr2=0x55c300520180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.264+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051fd00 0x55c300520180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccf77fe700 1 -- 172.21.15.145:0/1890631890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3002f05e0 con 0x55c30051f2e0 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009d80 tx=0x7fcce8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac42b07deba47e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccf7fff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 0x7fcce40a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fccf0094d70 tx=0x7fccf0021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccedffb700 1 -- 172.21.15.145:0/1890631890 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8004230 con 0x55c30051f2e0 2023-08-25T04:44:13.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccedffb700 1 -- 172.21.15.145:0/1890631890 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce8004390 con 0x55c30051f2e0 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.265+0000 7fccedffb700 1 -- 172.21.15.145:0/1890631890 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce80197c0 con 0x55c30051f2e0 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 -- 172.21.15.145:0/1890631890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 msgr2=0x55c30051f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009d80 tx=0x7fcce8011720 comp rx=0 tx=0).stop 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 -- 172.21.15.145:0/1890631890 shutdown_connections 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051fd00 0x55c300520180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c30051eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1890631890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051f2e0 0x55c30051f720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 -- 172.21.15.145:0/1890631890 >> 172.21.15.145:0/1890631890 conn(0x55c300493090 msgr2=0x55c300519250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 -- 172.21.15.145:0/1890631890 shutdown_connections 2023-08-25T04:44:13.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.266+0000 7fcd11432b80 1 -- 172.21.15.145:0/1890631890 wait complete. 2023-08-25T04:44:13.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 -- start start 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 Processor -- start 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300528350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c30052f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300470940 con 0x55c30051f2e0 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30040eb60 con 0x55c30051e980 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30040e860 con 0x55c30051fd00 2023-08-25T04:44:13.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.267+0000 7fcd11432b80 10 client.0 _ll_register_callbacks cb 0x55c300358a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:13.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fcceffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c30052f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fcceffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c30052f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40486/0 (socket says 172.21.15.145:40486) 2023-08-25T04:44:13.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60688/0 (socket says 172.21.15.145:60688) 2023-08-25T04:44:13.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fcceffff700 1 -- 172.21.15.145:0/3007221196 learned_addr learned my addr 172.21.15.145:0/3007221196 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fcceffff700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300528350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 -- 172.21.15.145:0/3007221196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 msgr2=0x55c30052f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c30052f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 -- 172.21.15.145:0/3007221196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 msgr2=0x55c300528350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300528350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 -- 172.21.15.145:0/3007221196 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce80099e0 con 0x55c30051f2e0 2023-08-25T04:44:13.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcce801b040 tx=0x7fcce8009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fb5813e6bab963 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.268+0000 7fccd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:13.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8003a90 con 0x55c30051f2e0 2023-08-25T04:44:13.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3004ee170 con 0x55c30051f2e0 2023-08-25T04:44:13.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcce8003bf0 con 0x55c30051f2e0 2023-08-25T04:44:13.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcce8005040 con 0x55c30051f2e0 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 10 client.15126 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c30029cd90 con 0x55c30051f2e0 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3004ee170 con 0x55c30051f2e0 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcce8021040 con 0x55c30051f2e0 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 10 client.15126 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 20 client.15126 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:13.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c300533ec0 0x55c300532980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 4 client.15126 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:13.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.269+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c300416b50 con 0x55c300533ec0 2023-08-25T04:44:13.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.270+0000 7fcceffff700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c300533ec0 0x55c300532980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.270+0000 7fccd6ffd700 10 client.15126 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:13.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.270+0000 7fcceffff700 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c300533ec0 0x55c300532980 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978a60f59da00e73 server_cookie=6053402cf1e8fd69 in_seq=0 out_seq=0 2023-08-25T04:44:13.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3007221196 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c300416b50 con 0x55c300533ec0 2023-08-25T04:44:13.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fccd6ffd700 10 client.15126 handle_command_reply: tid=0 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 client.15126 shutdown 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 msgr2=0x55c300528890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcce801b040 tx=0x7fcce8009f70 comp rx=0 tx=0).stop 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 shutdown_connections 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c300533ec0 0x55c300532980 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c30052f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fccd6ffd700 0 client.15126 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:13.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300528350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3007221196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c300528890 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 >> 172.21.15.145:0/3007221196 conn(0x55c300493090 msgr2=0x55c300513af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 shutdown_connections 2023-08-25T04:44:13.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.271+0000 7fcd11432b80 1 -- 172.21.15.145:0/3007221196 wait complete. 2023-08-25T04:44:13.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.272+0000 7fcd11432b80 20 client.15126 trim_cache size 0 max 16384 2023-08-25T04:44:13.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.289+0000 7fcd11432b80 1 Processor -- start 2023-08-25T04:44:13.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.289+0000 7fcd11432b80 1 -- start start 2023-08-25T04:44:13.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c300526010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051fd00 0x55c300522b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3002f28d0 con 0x55c30051f2e0 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30040ae00 con 0x55c30051fd00 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcd11432b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300515c80 con 0x55c30051e980 2023-08-25T04:44:13.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60694/0 (socket says 172.21.15.145:60694) 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 -- 172.21.15.145:0/1439824629 learned_addr learned my addr 172.21.15.145:0/1439824629 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcceffff700 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c300526010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fcceffff700 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051fd00 0x55c300522b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 -- 172.21.15.145:0/1439824629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 msgr2=0x55c300526010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c300526010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 -- 172.21.15.145:0/1439824629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051fd00 msgr2=0x55c300522b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051fd00 0x55c300522b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.290+0000 7fccf77fe700 1 -- 172.21.15.145:0/1439824629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c300285c10 con 0x55c30051f2e0 2023-08-25T04:44:13.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fccf77fe700 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcce80095f0 tx=0x7fcce8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861a95e65e665b95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fccee7fc700 1 -- 172.21.15.145:0/1439824629 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8005c20 con 0x55c30051f2e0 2023-08-25T04:44:13.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fccee7fc700 1 -- 172.21.15.145:0/1439824629 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce8004b20 con 0x55c30051f2e0 2023-08-25T04:44:13.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fccee7fc700 1 -- 172.21.15.145:0/1439824629 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8019ae0 con 0x55c30051f2e0 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fcd11432b80 1 -- 172.21.15.145:0/1439824629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 msgr2=0x55c30052f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.291+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcce80095f0 tx=0x7fcce8009260 comp rx=0 tx=0).stop 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 -- 172.21.15.145:0/1439824629 shutdown_connections 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051fd00 0x55c300522b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051e980 0x55c300526010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 --2- 172.21.15.145:0/1439824629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 -- 172.21.15.145:0/1439824629 >> 172.21.15.145:0/1439824629 conn(0x55c300493090 msgr2=0x55c300512ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 -- 172.21.15.145:0/1439824629 shutdown_connections 2023-08-25T04:44:13.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 -- 172.21.15.145:0/1439824629 wait complete. 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 -- start start 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.292+0000 7fcd11432b80 1 Processor -- start 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300524d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c3005252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300525890 con 0x55c30051f2e0 2023-08-25T04:44:13.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300513260 con 0x55c30051e980 2023-08-25T04:44:13.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c300513420 con 0x55c30051fd00 2023-08-25T04:44:13.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcd11432b80 10 client.0 _ll_register_callbacks cb 0x7fcce8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:13.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcceffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300524d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60702/0 (socket says 172.21.15.145:60702) 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 -- 172.21.15.145:0/3077501299 learned_addr learned my addr 172.21.15.145:0/3077501299 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcceffff700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c3005252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 -- 172.21.15.145:0/3077501299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 msgr2=0x55c3005252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c3005252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 -- 172.21.15.145:0/3077501299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 msgr2=0x55c300524d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:13.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300524d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 -- 172.21.15.145:0/3077501299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c30029cd90 con 0x55c30051f2e0 2023-08-25T04:44:13.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fcceffff700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c3005252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:13.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccf77fe700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcce8011760 tx=0x7fcce8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638f45eb983366f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:13.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.293+0000 7fccd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:13.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8004660 con 0x55c30051f2e0 2023-08-25T04:44:13.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c300511860 con 0x55c30051f2e0 2023-08-25T04:44:13.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcce80047c0 con 0x55c30051f2e0 2023-08-25T04:44:13.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcce80049a0 con 0x55c30051f2e0 2023-08-25T04:44:13.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fcd11432b80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:13.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c3002c3200 con 0x55c30051f2e0 2023-08-25T04:44:13.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c300511860 con 0x55c30051f2e0 2023-08-25T04:44:13.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcce8021070 con 0x55c30051f2e0 2023-08-25T04:44:13.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fcd11432b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:13.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.294+0000 7fcd11432b80 10 client.15132 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcd11432b80 20 client.15132 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c30058b730 0x55c300587c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcd11432b80 4 client.15132 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c300469920 con 0x55c30058b730 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcceffff700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c30058b730 0x55c300587c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fccd6ffd700 10 client.15132 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcceffff700 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c30058b730 0x55c300587c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb67d2e64cf2be25 server_cookie=e012a50ccfdcd317 in_seq=0 out_seq=0 2023-08-25T04:44:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fccd6ffd700 1 -- 172.21.15.145:0/3077501299 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c300469920 con 0x55c30058b730 2023-08-25T04:44:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fccd6ffd700 10 client.15132 handle_command_reply: tid=0 2023-08-25T04:44:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.295+0000 7fcd11432b80 1 client.15132 shutdown 2023-08-25T04:44:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 msgr2=0x55c30052f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcce8011760 tx=0x7fcce8005a00 comp rx=0 tx=0).stop 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 shutdown_connections 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c30058b730 0x55c300587c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30051fd00 0x55c3005252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c30051e980 0x55c300524d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fccd6ffd700 0 client.15132 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 --2- 172.21.15.145:0/3077501299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c30051f2e0 0x55c30052f7f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 >> 172.21.15.145:0/3077501299 conn(0x55c300493090 msgr2=0x55c300526010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 shutdown_connections 2023-08-25T04:44:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 1 -- 172.21.15.145:0/3077501299 wait complete. 2023-08-25T04:44:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fcd11432b80 20 client.15132 trim_cache size 0 max 16384 2023-08-25T04:44:13.311 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24562, 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:13.312 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 82.477035682999997, 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 92.722710090000007 2023-08-25T04:44:13.313 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:13.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 msgr2=0x7fcce40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 0x7fcce40a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fccf0094d70 tx=0x7fccf0021000 comp rx=0 tx=0).stop 2023-08-25T04:44:13.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 msgr2=0x7fccf819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:13.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.296+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef30 tx=0x7fccf000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 shutdown_connections 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcce409ff50 0x7fcce40a2410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fccf81087f0 0x7fccf819fe00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccf8073380 0x7fccf8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 --2- 172.21.15.145:0/1867738721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccf8074960 0x7fccf8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 >> 172.21.15.145:0/1867738721 conn(0x7fccf80fbea0 msgr2=0x7fccf80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 shutdown_connections 2023-08-25T04:44:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:13.297+0000 7fccfdbff700 1 -- 172.21.15.145:0/1867738721 wait complete. 2023-08-25T04:44:13.317 INFO:tasks.cephadm:Setup complete, yielding 2023-08-25T04:44:13.318 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-25T04:44:13.334 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:44:13.334 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch status' 2023-08-25T04:44:14.057 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:14 smithi145 ceph-mon[97681]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:14 smithi162 ceph-mon[130941]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:15.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:14 smithi183 ceph-mon[131414]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:15.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 -- 172.21.15.145:0/2155004344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b41009b0 msgr2=0x7f64b4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:15.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2155004344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b41009b0 0x7f64b4100dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64a40099c0 tx=0x7f64a4009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 -- 172.21.15.145:0/2155004344 shutdown_connections 2023-08-25T04:44:15.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2155004344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4102550 0x7f64b4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2155004344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b41009b0 0x7f64b4100dd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2155004344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b4101bb0 0x7f64b4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 -- 172.21.15.145:0/2155004344 >> 172.21.15.145:0/2155004344 conn(0x7f64b40fbf10 msgr2=0x7f64b40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:15.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 -- 172.21.15.145:0/2155004344 shutdown_connections 2023-08-25T04:44:15.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.353+0000 7f64ba84a700 1 -- 172.21.15.145:0/2155004344 wait complete. 2023-08-25T04:44:15.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 Processor -- start 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 -- start start 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4101bb0 0x7f64b4196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 0x7f64b419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b419e200 con 0x7f64b4102550 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b419e340 con 0x7f64b41009b0 2023-08-25T04:44:15.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.354+0000 7f64ba84a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b419e4d0 con 0x7f64b4101bb0 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 0x7f64b419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:34762/0 (socket says 172.21.15.145:34762) 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b3fff700 1 -- 172.21.15.145:0/2365113320 learned_addr learned my addr 172.21.15.145:0/2365113320 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b37fe700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4101bb0 0x7f64b4196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 -- 172.21.15.145:0/2365113320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4101bb0 msgr2=0x7f64b4196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:15.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4101bb0 0x7f64b4196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 -- 172.21.15.145:0/2365113320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 msgr2=0x7f64b4196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 -- 172.21.15.145:0/2365113320 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40095f0 con 0x7f64b4102550 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b3fff700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.355+0000 7f64b8de7700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 0x7f64b419dc30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009f60 tx=0x7f64ac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.356+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac018070 con 0x7f64b4102550 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.356+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b419e750 con 0x7f64b4102550 2023-08-25T04:44:15.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.356+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64ac00f040 con 0x7f64b4102550 2023-08-25T04:44:15.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.356+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00ccc0 con 0x7f64b4102550 2023-08-25T04:44:15.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.356+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b419ec50 con 0x7f64b4102550 2023-08-25T04:44:15.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.357+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b410c1b0 con 0x7f64b4102550 2023-08-25T04:44:15.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.359+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f64ac003a00 con 0x7f64b4102550 2023-08-25T04:44:15.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.360+0000 7f64b17fa700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 0x7f649c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:15.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.360+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f64ac0caff0 con 0x7f64b4102550 2023-08-25T04:44:15.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.360+0000 7f64b3fff700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 0x7f649c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:15.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.361+0000 7f64b3fff700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 0x7f649c0a2410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64a4000c00 tx=0x7f64a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:15.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.364+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f64ac0fde00 con 0x7f64b4102550 2023-08-25T04:44:15.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.580+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64b419ee70 con 0x7f649c09ff50 2023-08-25T04:44:15.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.581+0000 7f64b17fa700 1 -- 172.21.15.145:0/2365113320 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f64b419ee70 con 0x7f649c09ff50 2023-08-25T04:44:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:Backend: cephadm 2023-08-25T04:44:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:Available: Yes 2023-08-25T04:44:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:Paused: No 2023-08-25T04:44:15.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.583+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 msgr2=0x7f649c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:15.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.583+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 0x7f649c0a2410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64a4000c00 tx=0x7f64a4005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.583+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 msgr2=0x7f64b419dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.583+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 0x7f64b419dc30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009f60 tx=0x7f64ac009f90 comp rx=0 tx=0).stop 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 shutdown_connections 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f649c09ff50 0x7f649c0a2410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64b4102550 0x7f64b419dc30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64b41009b0 0x7f64b4196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 --2- 172.21.15.145:0/2365113320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64b4101bb0 0x7f64b4196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:15.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 >> 172.21.15.145:0/2365113320 conn(0x7f64b40fbf10 msgr2=0x7f64b40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:15.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 shutdown_connections 2023-08-25T04:44:15.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:15.584+0000 7f64ba84a700 1 -- 172.21.15.145:0/2365113320 wait complete. 2023-08-25T04:44:16.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch ps' 2023-08-25T04:44:16.773 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:17.060 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:16 smithi145 ceph-mon[97681]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:17.060 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:16 smithi145 ceph-mon[97681]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:16 smithi162 ceph-mon[130941]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:16 smithi162 ceph-mon[130941]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:16 smithi183 ceph-mon[131414]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:16 smithi183 ceph-mon[131414]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:18.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 -- 172.21.15.145:0/3265135864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 msgr2=0x7faaec100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 --2- 172.21.15.145:0/3265135864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec100dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faadc0099c0 tx=0x7faadc009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 -- 172.21.15.145:0/3265135864 shutdown_connections 2023-08-25T04:44:18.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 --2- 172.21.15.145:0/3265135864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec102550 0x7faaec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 --2- 172.21.15.145:0/3265135864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec100dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 --2- 172.21.15.145:0/3265135864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec101bb0 0x7faaec102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 -- 172.21.15.145:0/3265135864 >> 172.21.15.145:0/3265135864 conn(0x7faaec0fbf10 msgr2=0x7faaec0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.192+0000 7faaf293d700 1 -- 172.21.15.145:0/3265135864 shutdown_connections 2023-08-25T04:44:18.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.193+0000 7faaf293d700 1 -- 172.21.15.145:0/3265135864 wait complete. 2023-08-25T04:44:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.193+0000 7faaf293d700 1 Processor -- start 2023-08-25T04:44:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 -- start start 2023-08-25T04:44:18.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec101bb0 0x7faaec198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec102550 0x7faaec19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a0470 con 0x7faaec1009b0 2023-08-25T04:44:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a05b0 con 0x7faaec102550 2023-08-25T04:44:18.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf293d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a0740 con 0x7faaec101bb0 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaebfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaebfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60738/0 (socket says 172.21.15.145:60738) 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaebfff700 1 -- 172.21.15.145:0/561485857 learned_addr learned my addr 172.21.15.145:0/561485857 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaeb7fe700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec101bb0 0x7faaec198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.194+0000 7faaf0eda700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec102550 0x7faaec19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:18.199 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 -- 172.21.15.145:0/561485857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec101bb0 msgr2=0x7faaec198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec101bb0 0x7faaec198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 -- 172.21.15.145:0/561485857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec102550 msgr2=0x7faaec19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec102550 0x7faaec19fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 -- 172.21.15.145:0/561485857 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0095f0 con 0x7faaec1009b0 2023-08-25T04:44:18.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaebfff700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faadc00bf40 tx=0x7faadc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:18.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc01d070 con 0x7faaec1009b0 2023-08-25T04:44:18.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faadc005050 con 0x7faaec1009b0 2023-08-25T04:44:18.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec1a09c0 con 0x7faaec1009b0 2023-08-25T04:44:18.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc004b30 con 0x7faaec1009b0 2023-08-25T04:44:18.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.195+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec1a0e30 con 0x7faaec1009b0 2023-08-25T04:44:18.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.196+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec10c200 con 0x7faaec1009b0 2023-08-25T04:44:18.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.198+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7faadc0051c0 con 0x7faaec1009b0 2023-08-25T04:44:18.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.200+0000 7faae97fa700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 0x7faad40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:18.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.200+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faadc0cc380 con 0x7faaec1009b0 2023-08-25T04:44:18.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.200+0000 7faaeb7fe700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 0x7faad40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:18.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.200+0000 7faaeb7fe700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 0x7faad40a2490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faae0009c60 tx=0x7faae0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:18.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.204+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faadc095ea0 con 0x7faaec1009b0 2023-08-25T04:44:18.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.415+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faaec108550 con 0x7faad409ffd0 2023-08-25T04:44:18.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.427+0000 7faae97fa700 1 -- 172.21.15.145:0/561485857 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7faaec108550 con 0x7faad409ffd0 2023-08-25T04:44:18.429 INFO:teuthology.orchestra.run.smithi145.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-08-25T04:44:18.429 INFO:teuthology.orchestra.run.smithi145.stdout:mds.a smithi145 running (98s) - 97s 21.3M - 18.0.0-5703-gba6b39e4 95c9c69e1344 d41923750717 2023-08-25T04:44:18.429 INFO:teuthology.orchestra.run.smithi145.stdout:mds.b smithi162 running (95s) - 95s 17.7M - 18.0.0-5703-gba6b39e4 95c9c69e1344 70d60d983197 2023-08-25T04:44:18.429 INFO:teuthology.orchestra.run.smithi145.stdout:mds.c smithi183 running (108s) - 108s 18.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 aa26c26f98db 2023-08-25T04:44:18.429 INFO:teuthology.orchestra.run.smithi145.stdout:mds.d smithi145 running (115s) - 114s 13.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 77c21fb1862e 2023-08-25T04:44:18.430 INFO:teuthology.orchestra.run.smithi145.stdout:mds.e smithi162 running (112s) - 112s 14.8M - 18.0.0-5703-gba6b39e4 95c9c69e1344 6d9400e38ff3 2023-08-25T04:44:18.430 INFO:teuthology.orchestra.run.smithi145.stdout:mds.f smithi183 running (110s) - 110s 17.0M - 18.0.0-5703-gba6b39e4 95c9c69e1344 b85f8454c710 2023-08-25T04:44:18.430 INFO:teuthology.orchestra.run.smithi145.stdout:mds.g smithi145 running (101s) - 100s 13.4M - 18.0.0-5703-gba6b39e4 95c9c69e1344 b3464942652e 2023-08-25T04:44:18.430 INFO:teuthology.orchestra.run.smithi145.stdout:mds.h smithi162 running (106s) - 106s 17.2M - 18.0.0-5703-gba6b39e4 95c9c69e1344 921b5f9fb2af 2023-08-25T04:44:18.430 INFO:teuthology.orchestra.run.smithi145.stdout:mds.i smithi183 running (104s) - 103s 18.1M - 18.0.0-5703-gba6b39e4 95c9c69e1344 4119f03ccf67 2023-08-25T04:44:18.431 INFO:teuthology.orchestra.run.smithi145.stdout:mds.j smithi145 running (118s) - 117s 13.5M - 18.0.0-5703-gba6b39e4 95c9c69e1344 9d8be3e6fcec 2023-08-25T04:44:18.431 INFO:teuthology.orchestra.run.smithi145.stdout:mds.k smithi162 running (2m) - 2m 17.2M - 18.0.0-5703-gba6b39e4 95c9c69e1344 7b429094ee57 2023-08-25T04:44:18.431 INFO:teuthology.orchestra.run.smithi145.stdout:mds.l smithi183 running (2m) - 2m 16.1M - 18.0.0-5703-gba6b39e4 95c9c69e1344 176afc5069b6 2023-08-25T04:44:18.431 INFO:teuthology.orchestra.run.smithi145.stdout:mgr.x smithi145 *:9283,8765 running (8m) - 8m 524M - 18.0.0-5703-gba6b39e4 95c9c69e1344 dd74c19a970f 2023-08-25T04:44:18.432 INFO:teuthology.orchestra.run.smithi145.stdout:mgr.y smithi162 *:8443,8765 running (6m) - 6m 451M - 18.0.0-5703-gba6b39e4 95c9c69e1344 5324331668e7 2023-08-25T04:44:18.432 INFO:teuthology.orchestra.run.smithi145.stdout:mgr.z smithi183 *:8443,8765 running (6m) - 6m 449M - 18.0.0-5703-gba6b39e4 95c9c69e1344 5838764ef720 2023-08-25T04:44:18.432 INFO:teuthology.orchestra.run.smithi145.stdout:mon.a smithi145 running (8m) - 8m 94.1M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 60fb508e7675 2023-08-25T04:44:18.432 INFO:teuthology.orchestra.run.smithi145.stdout:mon.b smithi162 running (6m) - 6m 59.9M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 e0b99022b400 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:mon.c smithi183 running (7m) - 7m 60.7M 2048M 18.0.0-5703-gba6b39e4 95c9c69e1344 a4c7df1ffeca 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:osd.0 smithi145 running (6m) - 6m 71.1M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 ce718e8fdf8d 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:osd.1 smithi162 running (5m) - 5m 80.9M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 81aec886c2c4 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:osd.2 smithi183 running (5m) - 5m 73.7M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 8ce14eba472a 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:osd.3 smithi145 running (5m) - 5m 65.4M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 c94308da7866 2023-08-25T04:44:18.433 INFO:teuthology.orchestra.run.smithi145.stdout:osd.4 smithi162 running (4m) - 4m 64.1M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 67618ac84eae 2023-08-25T04:44:18.434 INFO:teuthology.orchestra.run.smithi145.stdout:osd.5 smithi183 running (4m) - 4m 67.9M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 4025f04ef6d5 2023-08-25T04:44:18.434 INFO:teuthology.orchestra.run.smithi145.stdout:osd.6 smithi145 running (4m) - 4m 63.2M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 d90f56ede1ab 2023-08-25T04:44:18.434 INFO:teuthology.orchestra.run.smithi145.stdout:osd.7 smithi162 running (3m) - 3m 66.5M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 c969e177f875 2023-08-25T04:44:18.434 INFO:teuthology.orchestra.run.smithi145.stdout:osd.8 smithi183 running (3m) - 3m 61.5M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 161e15014841 2023-08-25T04:44:18.435 INFO:teuthology.orchestra.run.smithi145.stdout:osd.9 smithi145 running (3m) - 3m 60.2M 4316M 18.0.0-5703-gba6b39e4 95c9c69e1344 676287f02a33 2023-08-25T04:44:18.435 INFO:teuthology.orchestra.run.smithi145.stdout:osd.10 smithi162 running (2m) - 2m 60.3M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 36ee5a97ed44 2023-08-25T04:44:18.435 INFO:teuthology.orchestra.run.smithi145.stdout:osd.11 smithi183 running (2m) - 2m 55.4M 4292M 18.0.0-5703-gba6b39e4 95c9c69e1344 e9a5b7fdf467 2023-08-25T04:44:18.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 msgr2=0x7faad40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:18.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 0x7faad40a2490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faae0009c60 tx=0x7faae0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:18.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 msgr2=0x7faaec1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:18.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faadc00bf40 tx=0x7faadc00f690 comp rx=0 tx=0).stop 2023-08-25T04:44:18.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 shutdown_connections 2023-08-25T04:44:18.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faad409ffd0 0x7faad40a2490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaec102550 0x7faaec19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaec1009b0 0x7faaec1988e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 --2- 172.21.15.145:0/561485857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec101bb0 0x7faaec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:18.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 >> 172.21.15.145:0/561485857 conn(0x7faaec0fbf10 msgr2=0x7faaec0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:18.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 shutdown_connections 2023-08-25T04:44:18.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:18.430+0000 7faaf293d700 1 -- 172.21.15.145:0/561485857 wait complete. 2023-08-25T04:44:18.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch ls' 2023-08-25T04:44:19.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:18 smithi145 ceph-mon[97681]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:18 smithi162 ceph-mon[130941]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:18 smithi183 ceph-mon[131414]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:19.590 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:19 smithi145 ceph-mon[97681]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:19 smithi162 ceph-mon[130941]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:19 smithi183 ceph-mon[131414]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:21.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 -- 172.21.15.145:0/3558772644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 msgr2=0x7f4384108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 --2- 172.21.15.145:0/3558772644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f4384108e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f437c00b0d0 tx=0x7f437c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 -- 172.21.15.145:0/3558772644 shutdown_connections 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 --2- 172.21.15.145:0/3558772644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f4384108e00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 --2- 172.21.15.145:0/3558772644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 0x7f4384100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 --2- 172.21.15.145:0/3558772644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 0x7f4384102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.001+0000 7f438c104700 1 -- 172.21.15.145:0/3558772644 >> 172.21.15.145:0/3558772644 conn(0x7f43840fbf10 msgr2=0x7f43840fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:21.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.002+0000 7f438c104700 1 -- 172.21.15.145:0/3558772644 shutdown_connections 2023-08-25T04:44:21.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.002+0000 7f438c104700 1 -- 172.21.15.145:0/3558772644 wait complete. 2023-08-25T04:44:21.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 Processor -- start 2023-08-25T04:44:21.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 -- start start 2023-08-25T04:44:21.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 0x7f4384198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:21.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 0x7f4384198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a0510 con 0x7f4384102550 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a0650 con 0x7f4384101bb0 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438c104700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841a07e0 con 0x7f43841009b0 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438a6a1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438a6a1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:60758/0 (socket says 172.21.15.145:60758) 2023-08-25T04:44:21.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f4389ea0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 0x7f4384198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438a6a1700 1 -- 172.21.15.145:0/1669195313 learned_addr learned my addr 172.21.15.145:0/1669195313 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.003+0000 7f438969f700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 0x7f4384198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 -- 172.21.15.145:0/1669195313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 msgr2=0x7f4384198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 0x7f4384198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 -- 172.21.15.145:0/1669195313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 msgr2=0x7f4384198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 0x7f4384198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 -- 172.21.15.145:0/1669195313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c009d00 con 0x7f4384102550 2023-08-25T04:44:21.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438a6a1700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f437c008ee0 tx=0x7f437c008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c00e040 con 0x7f4384102550 2023-08-25T04:44:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43841a0a60 con 0x7f4384102550 2023-08-25T04:44:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f437c00bdb0 con 0x7f4384102550 2023-08-25T04:44:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c01d470 con 0x7f4384102550 2023-08-25T04:44:21.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.004+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43841a0e80 con 0x7f4384102550 2023-08-25T04:44:21.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.006+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438410c210 con 0x7f4384102550 2023-08-25T04:44:21.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.007+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f437c0050b0 con 0x7f4384102550 2023-08-25T04:44:21.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.008+0000 7f437affd700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 0x7f43700a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:21.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.009+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f437c0cc7a0 con 0x7f4384102550 2023-08-25T04:44:21.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.009+0000 7f4389ea0700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 0x7f43700a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:21.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.009+0000 7f4389ea0700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 0x7f43700a2410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4384101a10 tx=0x7f4380009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:21.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.013+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f437c096260 con 0x7f4384102550 2023-08-25T04:44:21.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:20 smithi145 ceph-mon[97681]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:21.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:20 smithi162 ceph-mon[130941]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:21.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:20 smithi183 ceph-mon[131414]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:21.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.220+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4384108b50 con 0x7f437009ff50 2023-08-25T04:44:21.226 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.225+0000 7f437affd700 1 -- 172.21.15.145:0/1669195313 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4384108b50 con 0x7f437009ff50 2023-08-25T04:44:21.227 INFO:teuthology.orchestra.run.smithi145.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-08-25T04:44:21.227 INFO:teuthology.orchestra.run.smithi145.stdout:mds.all 12/12 - 2m smithi145=a;smithi145=d;smithi145=g;smithi145=j;smithi162=b;smithi162=e;smithi162=h;smithi162=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2023-08-25T04:44:21.227 INFO:teuthology.orchestra.run.smithi145.stdout:mgr 3/3 - 6m smithi145=x;smithi162=y;smithi183=z;count:3 2023-08-25T04:44:21.227 INFO:teuthology.orchestra.run.smithi145.stdout:mon 3/3 - 7m smithi145:172.21.15.145=a;smithi162:172.21.15.162=b;smithi183:172.21.15.183=c;count:3 2023-08-25T04:44:21.228 INFO:teuthology.orchestra.run.smithi145.stdout:osd 12 - - 2023-08-25T04:44:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 msgr2=0x7f43700a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 0x7f43700a2410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4384101a10 tx=0x7f4380009260 comp rx=0 tx=0).stop 2023-08-25T04:44:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 msgr2=0x7f438419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f437c008ee0 tx=0x7f437c008f10 comp rx=0 tx=0).stop 2023-08-25T04:44:21.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 shutdown_connections 2023-08-25T04:44:21.230 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f437009ff50 0x7f43700a2410 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.230 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4384102550 0x7f438419ff40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.230 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841009b0 0x7f4384198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.230 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 --2- 172.21.15.145:0/1669195313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4384101bb0 0x7f4384198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:21.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 >> 172.21.15.145:0/1669195313 conn(0x7f43840fbf10 msgr2=0x7f4384107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:21.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 shutdown_connections 2023-08-25T04:44:21.231 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:21.228+0000 7f438c104700 1 -- 172.21.15.145:0/1669195313 wait complete. 2023-08-25T04:44:21.677 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch host ls' 2023-08-25T04:44:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:22.901 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:22 smithi145 ceph-mon[97681]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:22.902 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:22 smithi145 ceph-mon[97681]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:23.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:22 smithi162 ceph-mon[130941]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:23.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:22 smithi162 ceph-mon[130941]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:22 smithi183 ceph-mon[131414]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:22 smithi183 ceph-mon[131414]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:23.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 -- 172.21.15.145:0/1656212120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 msgr2=0x7f9150100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:23.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 --2- 172.21.15.145:0/1656212120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150100dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9140009920 tx=0x7f9140009c30 comp rx=0 tx=0).stop 2023-08-25T04:44:23.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 -- 172.21.15.145:0/1656212120 shutdown_connections 2023-08-25T04:44:23.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 --2- 172.21.15.145:0/1656212120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150102550 0x7f9150108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 --2- 172.21.15.145:0/1656212120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150100dd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 --2- 172.21.15.145:0/1656212120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150101bb0 0x7f9150102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 -- 172.21.15.145:0/1656212120 >> 172.21.15.145:0/1656212120 conn(0x7f91500fbf10 msgr2=0x7f91500fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:23.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 -- 172.21.15.145:0/1656212120 shutdown_connections 2023-08-25T04:44:23.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.661+0000 7f9155fbf700 1 -- 172.21.15.145:0/1656212120 wait complete. 2023-08-25T04:44:23.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.663+0000 7f9155fbf700 1 Processor -- start 2023-08-25T04:44:23.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.663+0000 7f9155fbf700 1 -- start start 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150102550 0x7f915019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501a04b0 con 0x7f9150101bb0 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501a05f0 con 0x7f91501009b0 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f9155fbf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501a0780 con 0x7f9150102550 2023-08-25T04:44:23.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40164/0 (socket says 172.21.15.145:40164) 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 -- 172.21.15.145:0/24870959 learned_addr learned my addr 172.21.15.145:0/24870959 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914ffff700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150102550 0x7f915019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 -- 172.21.15.145:0/24870959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150102550 msgr2=0x7f915019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150102550 0x7f915019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 -- 172.21.15.145:0/24870959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 msgr2=0x7f9150198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:23.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.664+0000 7f914effd700 1 -- 172.21.15.145:0/24870959 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140009520 con 0x7f9150101bb0 2023-08-25T04:44:23.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f914effd700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:23.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144004a30 con 0x7f9150101bb0 2023-08-25T04:44:23.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91501a0a60 con 0x7f9150101bb0 2023-08-25T04:44:23.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91440054e0 con 0x7f9150101bb0 2023-08-25T04:44:23.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400cc80 con 0x7f9150101bb0 2023-08-25T04:44:23.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.665+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91501a0fb0 con 0x7f9150101bb0 2023-08-25T04:44:23.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.666+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915010c1b0 con 0x7f9150101bb0 2023-08-25T04:44:23.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.668+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9144005000 con 0x7f9150101bb0 2023-08-25T04:44:23.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.670+0000 7f914cff9700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 0x7f91380a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:23.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.670+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9144014070 con 0x7f9150101bb0 2023-08-25T04:44:23.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.670+0000 7f914f7fe700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 0x7f91380a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:23.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.670+0000 7f914f7fe700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 0x7f91380a2490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91400098f0 tx=0x7f9140019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:23.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.672+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f91440954c0 con 0x7f9150101bb0 2023-08-25T04:44:23.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.880+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f91501085e0 con 0x7f913809ffd0 2023-08-25T04:44:23.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.881+0000 7f914cff9700 1 -- 172.21.15.145:0/24870959 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f91501085e0 con 0x7f913809ffd0 2023-08-25T04:44:23.882 INFO:teuthology.orchestra.run.smithi145.stdout:HOST ADDR LABELS STATUS 2023-08-25T04:44:23.882 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145 172.21.15.145 2023-08-25T04:44:23.882 INFO:teuthology.orchestra.run.smithi145.stdout:smithi162 172.21.15.162 2023-08-25T04:44:23.882 INFO:teuthology.orchestra.run.smithi145.stdout:smithi183 172.21.15.183 2023-08-25T04:44:23.882 INFO:teuthology.orchestra.run.smithi145.stdout:3 hosts in cluster 2023-08-25T04:44:23.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 msgr2=0x7f91380a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 0x7f91380a2490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91400098f0 tx=0x7f9140019040 comp rx=0 tx=0).stop 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 msgr2=0x7f9150198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400ed00 comp rx=0 tx=0).stop 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 shutdown_connections 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f913809ffd0 0x7f91380a2490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9150102550 0x7f915019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91501009b0 0x7f9150198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 --2- 172.21.15.145:0/24870959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9150101bb0 0x7f9150198e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 >> 172.21.15.145:0/24870959 conn(0x7f91500fbf10 msgr2=0x7f91500fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 shutdown_connections 2023-08-25T04:44:23.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:23.884+0000 7f9155fbf700 1 -- 172.21.15.145:0/24870959 wait complete. 2023-08-25T04:44:24.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph orch device ls' 2023-08-25T04:44:25.016 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:25.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:24 smithi145 ceph-mon[97681]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:25.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:24 smithi145 ceph-mon[97681]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:25.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:24 smithi162 ceph-mon[130941]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:24 smithi162 ceph-mon[130941]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:24 smithi183 ceph-mon[131414]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:24 smithi183 ceph-mon[131414]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:26.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 -- 172.21.15.145:0/4052281466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102550 msgr2=0x7f17c0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:26.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 --2- 172.21.15.145:0/4052281466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102550 0x7f17c0108e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b0d0 tx=0x7f17b800b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 -- 172.21.15.145:0/4052281466 shutdown_connections 2023-08-25T04:44:26.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 --2- 172.21.15.145:0/4052281466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c0102550 0x7f17c0108e00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 --2- 172.21.15.145:0/4052281466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c01009b0 0x7f17c0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 --2- 172.21.15.145:0/4052281466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 -- 172.21.15.145:0/4052281466 >> 172.21.15.145:0/4052281466 conn(0x7f17c00fbf10 msgr2=0x7f17c00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:26.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 -- 172.21.15.145:0/4052281466 shutdown_connections 2023-08-25T04:44:26.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.559+0000 7f17c5422700 1 -- 172.21.15.145:0/4052281466 wait complete. 2023-08-25T04:44:26.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.560+0000 7f17c5422700 1 Processor -- start 2023-08-25T04:44:26.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.560+0000 7f17c5422700 1 -- start start 2023-08-25T04:44:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.560+0000 7f17c5422700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 0x7f17c01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.560+0000 7f17c5422700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.560+0000 7f17c5422700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c0102550 0x7f17c019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17c5422700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019e330 con 0x7f17c0101bb0 2023-08-25T04:44:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17c5422700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019e470 con 0x7f17c01009b0 2023-08-25T04:44:26.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17c5422700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019e600 con 0x7f17c0102550 2023-08-25T04:44:26.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:26.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40186/0 (socket says 172.21.15.145:40186) 2023-08-25T04:44:26.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 -- 172.21.15.145:0/1496901348 learned_addr learned my addr 172.21.15.145:0/1496901348 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:26.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17beffd700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 0x7f17c01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:26.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17bf7fe700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c0102550 0x7f17c019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:26.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 -- 172.21.15.145:0/1496901348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c0102550 msgr2=0x7f17c019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:26.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c0102550 0x7f17c019dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 -- 172.21.15.145:0/1496901348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 msgr2=0x7f17c01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 0x7f17c01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17be7fc700 1 -- 172.21.15.145:0/1496901348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8009d00 con 0x7f17c0101bb0 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.561+0000 7f17beffd700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 0x7f17c01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17be7fc700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17b400e9d0 tx=0x7f17b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4009790 con 0x7f17c0101bb0 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c019e8e0 con 0x7f17c0101bb0 2023-08-25T04:44:26.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b400ee70 con 0x7f17c0101bb0 2023-08-25T04:44:26.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4010ea0 con 0x7f17c0101bb0 2023-08-25T04:44:26.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.562+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c019ede0 con 0x7f17c0101bb0 2023-08-25T04:44:26.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.563+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0107f50 con 0x7f17c0101bb0 2023-08-25T04:44:26.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.565+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f17b400cbd0 con 0x7f17c0101bb0 2023-08-25T04:44:26.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.566+0000 7f17a7fff700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 0x7f17a80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:26.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.566+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f17b4014070 con 0x7f17c0101bb0 2023-08-25T04:44:26.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.567+0000 7f17beffd700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 0x7f17a80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:26.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.567+0000 7f17beffd700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 0x7f17a80a2410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f17c0101a10 tx=0x7f17b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:26.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.570+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f17b4051e00 con 0x7f17c0101bb0 2023-08-25T04:44:26.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.777+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c002ce20 con 0x7f17a809ff50 2023-08-25T04:44:26.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.779+0000 7f17a7fff700 1 -- 172.21.15.145:0/1496901348 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f17c002ce20 con 0x7f17a809ff50 2023-08-25T04:44:26.780 INFO:teuthology.orchestra.run.smithi145.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-08-25T04:44:26.780 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 3m ago LVM detected, locked 2023-08-25T04:44:26.780 INFO:teuthology.orchestra.run.smithi145.stdout:smithi162 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JU400BGN 372G No 2m ago LVM detected, locked 2023-08-25T04:44:26.780 INFO:teuthology.orchestra.run.smithi145.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago LVM detected, locked 2023-08-25T04:44:26.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.781+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 msgr2=0x7f17a80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:26.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.781+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 0x7f17a80a2410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f17c0101a10 tx=0x7f17b0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:26.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.781+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 msgr2=0x7f17c0196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.781+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17b400e9d0 tx=0x7f17b400ece0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.781+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 shutdown_connections 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f17a809ff50 0x7f17a80a2410 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c0102550 0x7f17c019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17c01009b0 0x7f17c01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 --2- 172.21.15.145:0/1496901348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c0101bb0 0x7f17c0196ce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:26.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 >> 172.21.15.145:0/1496901348 conn(0x7f17c00fbf10 msgr2=0x7f17c0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:26.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 shutdown_connections 2023-08-25T04:44:26.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:26.782+0000 7f17c5422700 1 -- 172.21.15.145:0/1496901348 wait complete. 2023-08-25T04:44:27.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:26 smithi145 ceph-mon[97681]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:27.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:26 smithi162 ceph-mon[130941]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:27.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:26 smithi183 ceph-mon[131414]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:27.305 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-25T04:44:27.319 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:44:27.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph fs dump' 2023-08-25T04:44:28.043 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:28.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:27 smithi145 ceph-mon[97681]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:27 smithi162 ceph-mon[130941]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:27 smithi183 ceph-mon[131414]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:44:28.914 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:28 smithi145 ceph-mon[97681]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:29.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:28 smithi162 ceph-mon[130941]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:29.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:28 smithi183 ceph-mon[131414]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:29.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.591+0000 7fd659cb6700 1 -- 172.21.15.145:0/3543510653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 msgr2=0x7fd65410ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:29.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.591+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3543510653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65410ad10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00b0a0 tx=0x7fd64c00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.591+0000 7fd659cb6700 1 -- 172.21.15.145:0/3543510653 shutdown_connections 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.591+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3543510653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65410ad10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.591+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3543510653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654074950 0x7fd654072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.592+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3543510653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654073380 0x7fd654073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.592+0000 7fd659cb6700 1 -- 172.21.15.145:0/3543510653 >> 172.21.15.145:0/3543510653 conn(0x7fd6540fbf10 msgr2=0x7fd6540fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.592+0000 7fd659cb6700 1 -- 172.21.15.145:0/3543510653 shutdown_connections 2023-08-25T04:44:29.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.592+0000 7fd659cb6700 1 -- 172.21.15.145:0/3543510653 wait complete. 2023-08-25T04:44:29.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 Processor -- start 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 -- start start 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654073380 0x7fd6541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654074950 0x7fd654198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541a0550 con 0x7fd654108880 2023-08-25T04:44:29.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541a0690 con 0x7fd654074950 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd659cb6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541a0820 con 0x7fd654073380 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.593+0000 7fd653fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd652ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654074950 0x7fd654198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40204/0 (socket says 172.21.15.145:40204) 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 -- 172.21.15.145:0/3375656240 learned_addr learned my addr 172.21.15.145:0/3375656240 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd6537fe700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654073380 0x7fd6541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:29.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 -- 172.21.15.145:0/3375656240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654073380 msgr2=0x7fd6541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:29.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654073380 0x7fd6541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 -- 172.21.15.145:0/3375656240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654074950 msgr2=0x7fd654198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:29.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654074950 0x7fd654198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 -- 172.21.15.145:0/3375656240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c009d00 con 0x7fd654108880 2023-08-25T04:44:29.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd653fff700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00b070 tx=0x7fd64c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:29.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00e040 con 0x7fd654108880 2023-08-25T04:44:29.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64c003e10 con 0x7fd654108880 2023-08-25T04:44:29.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c01b6c0 con 0x7fd654108880 2023-08-25T04:44:29.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.594+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6541a0aa0 con 0x7fd654108880 2023-08-25T04:44:29.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.595+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6541a0e60 con 0x7fd654108880 2023-08-25T04:44:29.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.596+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654106f30 con 0x7fd654108880 2023-08-25T04:44:29.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.597+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd64c01d470 con 0x7fd654108880 2023-08-25T04:44:29.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.599+0000 7fd650ff9700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 0x7fd63c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:29.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.599+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd64c0cc630 con 0x7fd654108880 2023-08-25T04:44:29.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.599+0000 7fd6537fe700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 0x7fd63c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:29.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.599+0000 7fd6537fe700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 0x7fd63c0a2490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd654074d30 tx=0x7fd644009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:29.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.603+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd64c0960f0 con 0x7fd654108880 2023-08-25T04:44:29.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.847+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd65410e440 con 0x7fd654108880 2023-08-25T04:44:29.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.848+0000 7fd650ff9700 1 -- 172.21.15.145:0/3375656240 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2956 (secure 0 0 0) 0x7fd64c099710 con 0x7fd654108880 2023-08-25T04:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:e23 2023-08-25T04:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-08-25T04:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T04:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:legacy client fscid: 1 2023-08-25T04:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:Filesystem 'cephfs' (1) 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:fs_name cephfs 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 23 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:created 2023-08-25T04:42:49.862390+0000 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:modified 2023-08-25T04:43:16.556417+0000 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:tableserver 0 2023-08-25T04:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:root 0 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:session_timeout 300 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:session_autoclose 300 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:max_file_size 1099511627776 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:max_xattr_size 65536 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:required_client_features {} 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:last_failure 0 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:last_failure_osd_epoch 0 2023-08-25T04:44:29.851 INFO:teuthology.orchestra.run.smithi145.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:max_mds 3 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:in 0,1,2 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:up {0=24562,1=24557,2=24532} 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:failed 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:damaged 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:stopped 2023-08-25T04:44:29.852 INFO:teuthology.orchestra.run.smithi145.stdout:data_pools [3,4] 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:metadata_pool 2 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:inline_data disabled 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:balancer 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:bal_rank_mask -1 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:standby_count_wanted 0 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a{0:24562} state up:active seq 4 addr [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.h{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.162:6836/1048214793,v1:172.21.15.162:6837/1048214793] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.853 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b{1:24557} state up:active seq 9 addr [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c{1:24517} state up:standby-replay seq 1 addr [v2:172.21.15.183:6836/4241662550,v1:172.21.15.183:6837/4241662550] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.i{2:24532} state up:active seq 12 addr [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.f{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/557521219,v1:172.21.15.183:6835/557521219] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout:Standby daemons: 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:29.854 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.j{-1:14454} state up:standby seq 1 addr [v2:172.21.15.145:6834/4172912052,v1:172.21.15.145:6835/4172912052] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.855 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d{-1:14466} state up:standby seq 1 addr [v2:172.21.15.145:6836/4182229769,v1:172.21.15.145:6837/4182229769] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.855 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.g{-1:14520} state up:standby seq 1 addr [v2:172.21.15.145:6838/3652134728,v1:172.21.15.145:6839/3652134728] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.855 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.162:6832/4170975809,v1:172.21.15.162:6833/4170975809] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.855 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.183:6832/916411904,v1:172.21.15.183:6833/916411904] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.855 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.e{-1:24488} state up:standby seq 1 addr [v2:172.21.15.162:6834/2094921807,v1:172.21.15.162:6835/2094921807] compat {c=[1],r=[1],i=[fff]}] 2023-08-25T04:44:29.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 msgr2=0x7fd63c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:29.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 0x7fd63c0a2490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd654074d30 tx=0x7fd644009260 comp rx=0 tx=0).stop 2023-08-25T04:44:29.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 msgr2=0x7fd65419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:29.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00b070 tx=0x7fd64c00bce0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 shutdown_connections 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd63c09ffd0 0x7fd63c0a2490 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd654108880 0x7fd65419ff80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd654073380 0x7fd6541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 --2- 172.21.15.145:0/3375656240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd654074950 0x7fd654198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 >> 172.21.15.145:0/3375656240 conn(0x7fd6540fbf10 msgr2=0x7fd6540fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 shutdown_connections 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:29.850+0000 7fd659cb6700 1 -- 172.21.15.145:0/3375656240 wait complete. 2023-08-25T04:44:29.857 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:30.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- bash -c 'ceph osd dump' 2023-08-25T04:44:30.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:30 smithi145 ceph-mon[97681]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:30.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:30 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3375656240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T04:44:30.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:30 smithi162 ceph-mon[130941]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:30.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:30 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3375656240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T04:44:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:30 smithi183 ceph-mon[131414]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:30 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3375656240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-25T04:44:30.967 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:32.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 -- 172.21.15.145:0/3658495924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 msgr2=0x7fc470102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:32.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 --2- 172.21.15.145:0/3658495924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc470102ff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0099c0 tx=0x7fc46c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 -- 172.21.15.145:0/3658495924 shutdown_connections 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 --2- 172.21.15.145:0/3658495924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470104770 0x7fc47010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 --2- 172.21.15.145:0/3658495924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc470102ff0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 --2- 172.21.15.145:0/3658495924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470103dd0 0x7fc470104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 -- 172.21.15.145:0/3658495924 >> 172.21.15.145:0/3658495924 conn(0x7fc4700fe130 msgr2=0x7fc4701005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 -- 172.21.15.145:0/3658495924 shutdown_connections 2023-08-25T04:44:32.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.443+0000 7fc4775ff700 1 -- 172.21.15.145:0/3658495924 wait complete. 2023-08-25T04:44:32.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.444+0000 7fc4775ff700 1 Processor -- start 2023-08-25T04:44:32.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.444+0000 7fc4775ff700 1 -- start start 2023-08-25T04:44:32.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470103dd0 0x7fc470198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 0x7fc47019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a0480 con 0x7fc470102bd0 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a05c0 con 0x7fc470103dd0 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc4775ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a0750 con 0x7fc470104770 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:32.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40218/0 (socket says 172.21.15.145:40218) 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 -- 172.21.15.145:0/2965276587 learned_addr learned my addr 172.21.15.145:0/2965276587 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc474b9a700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470103dd0 0x7fc470198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc475b9c700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 0x7fc47019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 -- 172.21.15.145:0/2965276587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 msgr2=0x7fc47019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 0x7fc47019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 -- 172.21.15.145:0/2965276587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470103dd0 msgr2=0x7fc470198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:32.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470103dd0 0x7fc470198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc47539b700 1 -- 172.21.15.145:0/2965276587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0095f0 con 0x7fc470102bd0 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.445+0000 7fc475b9c700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 0x7fc47019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc47539b700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00bf40 tx=0x7fc46c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c01d070 con 0x7fc470102bd0 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46c005050 con 0x7fc470102bd0 2023-08-25T04:44:32.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4701a09d0 con 0x7fc470102bd0 2023-08-25T04:44:32.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c004bd0 con 0x7fc470102bd0 2023-08-25T04:44:32.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.446+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4701a0e40 con 0x7fc470102bd0 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.447+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47010a240 con 0x7fc470102bd0 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.449+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fc46c0051c0 con 0x7fc470102bd0 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.450+0000 7fc4667fc700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 0x7fc45c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.450+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc46c0cc3c0 con 0x7fc470102bd0 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.451+0000 7fc474b9a700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 0x7fc45c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:32.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.451+0000 7fc474b9a700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 0x7fc45c0a2490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc460009c60 tx=0x7fc460009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:32.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.454+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc46c095ee0 con 0x7fc470102bd0 2023-08-25T04:44:32.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.654+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc47010e420 con 0x7fc470102bd0 2023-08-25T04:44:32.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.655+0000 7fc4667fc700 1 -- 172.21.15.145:0/2965276587 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) v1 ==== 56+0+5600 (secure 0 0 0) 0x7fc46c099500 con 0x7fc470102bd0 2023-08-25T04:44:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 89 2023-08-25T04:44:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:fsid b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T04:44:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:created 2023-08-25T04:35:34.328182+0000 2023-08-25T04:44:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:modified 2023-08-25T04:43:34.757361+0000 2023-08-25T04:44:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:crush_version 27 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:full_ratio 0.95 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:backfillfull_ratio 0.9 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:nearfull_ratio 0.85 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:require_min_compat_client luminous 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:min_compat_client luminous 2023-08-25T04:44:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:require_osd_release reef 2023-08-25T04:44:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:stretch_mode_enabled false 2023-08-25T04:44:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-08-25T04:44:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-08-25T04:44:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-08-25T04:44:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-08-25T04:44:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:max_osd 12 2023-08-25T04:44:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6802/3430955976,v1:172.21.15.145:6803/3430955976] [v2:172.21.15.145:6804/3430955976,v1:172.21.15.145:6805/3430955976] exists,up a6b0e0c3-39e3-4870-b353-a2e7c3811368 2023-08-25T04:44:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/4196277224,v1:172.21.15.162:6801/4196277224] [v2:172.21.15.162:6802/4196277224,v1:172.21.15.162:6803/4196277224] exists,up 825a87ec-1ec6-46eb-9e65-2aed8e37f8ae 2023-08-25T04:44:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/3244881082,v1:172.21.15.183:6801/3244881082] [v2:172.21.15.183:6802/3244881082,v1:172.21.15.183:6803/3244881082] exists,up 341bc527-6a20-413a-b07c-633058761811 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.3 up in weight 1 up_from 26 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6810/3785323151,v1:172.21.15.145:6811/3785323151] [v2:172.21.15.145:6812/3785323151,v1:172.21.15.145:6813/3785323151] exists,up d77c0b06-c1f7-4af2-ab8b-77c462d5cd39 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/739956893,v1:172.21.15.162:6809/739956893] [v2:172.21.15.162:6810/739956893,v1:172.21.15.162:6811/739956893] exists,up f547bc71-6bd5-4882-baa1-d043181e9ff8 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/15251435,v1:172.21.15.183:6809/15251435] [v2:172.21.15.183:6810/15251435,v1:172.21.15.183:6811/15251435] exists,up bffd6e2e-e1b7-4dee-8a67-98c02dc1162d 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6818/141129064,v1:172.21.15.145:6819/141129064] [v2:172.21.15.145:6820/141129064,v1:172.21.15.145:6821/141129064] exists,up 46c86644-5bd6-4012-8847-8e5c35e97f6c 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.7 up in weight 1 up_from 51 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/3155030948,v1:172.21.15.162:6817/3155030948] [v2:172.21.15.162:6818/3155030948,v1:172.21.15.162:6819/3155030948] exists,up 94da753b-5dbe-4a36-90c9-6d8d68d9d70b 2023-08-25T04:44:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:osd.8 up in weight 1 up_from 58 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3911543971,v1:172.21.15.183:6817/3911543971] [v2:172.21.15.183:6818/3911543971,v1:172.21.15.183:6819/3911543971] exists,up 577926f0-d4e5-4c92-9817-38ba81609f92 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:osd.9 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6826/4228444400,v1:172.21.15.145:6827/4228444400] [v2:172.21.15.145:6828/4228444400,v1:172.21.15.145:6829/4228444400] exists,up 0fc38b1d-c461-49db-a3cc-64ba6780cf5d 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:osd.10 up in weight 1 up_from 71 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/3093586037,v1:172.21.15.162:6825/3093586037] [v2:172.21.15.162:6826/3093586037,v1:172.21.15.162:6827/3093586037] exists,up a0aa5c31-938d-470f-bbba-9d2237cddae4 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:osd.11 up in weight 1 up_from 78 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1108166266,v1:172.21.15.183:6825/1108166266] [v2:172.21.15.183:6826/1108166266,v1:172.21.15.183:6827/1108166266] exists,up b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 2.6 [8,11] 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 2.a [2,11] 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 3.0 [5,9] 2023-08-25T04:44:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 3.3 [4,9] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 3.23 [4,8] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 3.2c [4,9] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 4.e [9,11,5,8] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 4.13 [9,8] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:pg_upmap_items 4.18 [9,8] 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/4198991683 expires 2023-08-26T04:36:33.606385+0000 2023-08-25T04:44:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/1908809275 expires 2023-08-26T04:36:33.606385+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:6801/3601102351 expires 2023-08-26T04:36:33.606385+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/227013713 expires 2023-08-26T04:36:06.999890+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/417312367 expires 2023-08-26T04:36:33.606385+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/1961008949 expires 2023-08-26T04:36:06.999890+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:0/3102359690 expires 2023-08-26T04:36:06.999890+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:6800/3601102351 expires 2023-08-26T04:36:33.606385+0000 2023-08-25T04:44:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:6800/2347349168 expires 2023-08-26T04:36:06.999890+0000 2023-08-25T04:44:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:blocklist 172.21.15.145:6801/2347349168 expires 2023-08-26T04:36:06.999890+0000 2023-08-25T04:44:32.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 msgr2=0x7fc45c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 0x7fc45c0a2490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc460009c60 tx=0x7fc460009260 comp rx=0 tx=0).stop 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 msgr2=0x7fc4701988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00bf40 tx=0x7fc46c00f690 comp rx=0 tx=0).stop 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 shutdown_connections 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fc45c09ffd0 0x7fc45c0a2490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc470104770 0x7fc47019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc470102bd0 0x7fc4701988f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 --2- 172.21.15.145:0/2965276587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc470103dd0 0x7fc470198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:32.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 >> 172.21.15.145:0/2965276587 conn(0x7fc4700fe130 msgr2=0x7fc470107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:32.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 shutdown_connections 2023-08-25T04:44:32.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:32.657+0000 7fc4775ff700 1 -- 172.21.15.145:0/2965276587 wait complete. 2023-08-25T04:44:33.054 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:32 smithi145 ceph-mon[97681]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:33.054 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:32 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2965276587' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T04:44:33.057 INFO:teuthology.run_tasks:Running task fs.ready... 2023-08-25T04:44:33.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:32 smithi162 ceph-mon[130941]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:33.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:32 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2965276587' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T04:44:33.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:32 smithi183 ceph-mon[131414]: pgmap v318: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:33.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:32 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2965276587' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-25T04:44:33.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 -- 172.21.15.145:0/3090623696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4604101b20 msgr2=0x7f4604101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 --2- 172.21.15.145:0/3090623696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4604101b20 0x7f4604101f80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0099c0 tx=0x7f45ec009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 -- 172.21.15.145:0/3090623696 shutdown_connections 2023-08-25T04:44:33.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 --2- 172.21.15.145:0/3090623696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46041024c0 0x7f4604108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 --2- 172.21.15.145:0/3090623696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 0x7f4604100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 --2- 172.21.15.145:0/3090623696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4604101b20 0x7f4604101f80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 -- 172.21.15.145:0/3090623696 >> 172.21.15.145:0/3090623696 conn(0x7f46040fbe60 msgr2=0x7f46040fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:33.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 -- 172.21.15.145:0/3090623696 shutdown_connections 2023-08-25T04:44:33.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.272+0000 7f460a5ff700 1 -- 172.21.15.145:0/3090623696 wait complete. 2023-08-25T04:44:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.273+0000 7f460a5ff700 1 Processor -- start 2023-08-25T04:44:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.273+0000 7f460a5ff700 1 -- start start 2023-08-25T04:44:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.273+0000 7f460a5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 0x7f46041965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.273+0000 7f460a5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4604101b20 0x7f4604196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f460a5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f460a5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419e150 con 0x7f46041024c0 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f460a5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419e290 con 0x7f4604101b20 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f460a5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419e420 con 0x7f4604100930 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49852/0 (socket says 172.21.15.145:49852) 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 -- 172.21.15.145:0/1718429743 learned_addr learned my addr 172.21.15.145:0/1718429743 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:33.277 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4603fff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 0x7f46041965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f45fbfff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4604101b20 0x7f4604196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 -- 172.21.15.145:0/1718429743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 msgr2=0x7f46041965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 0x7f46041965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 -- 172.21.15.145:0/1718429743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4604101b20 msgr2=0x7f4604196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4604101b20 0x7f4604196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 -- 172.21.15.145:0/1718429743 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec0095f0 con 0x7f46041024c0 2023-08-25T04:44:33.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.274+0000 7f4608b9c700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007ee0 tx=0x7f45fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:33.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.275+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc017070 con 0x7f46041024c0 2023-08-25T04:44:33.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.275+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45fc00f040 con 0x7f46041024c0 2023-08-25T04:44:33.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.275+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460419e700 con 0x7f46041024c0 2023-08-25T04:44:33.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.275+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc005580 con 0x7f46041024c0 2023-08-25T04:44:33.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.275+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460419ebc0 con 0x7f46041024c0 2023-08-25T04:44:33.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.276+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460410c190 con 0x7f46041024c0 2023-08-25T04:44:33.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.278+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f45fc00db20 con 0x7f46041024c0 2023-08-25T04:44:33.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.279+0000 7f4601ffb700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 0x7f45f00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.279+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f45fc0cb350 con 0x7f46041024c0 2023-08-25T04:44:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.279+0000 7f4603fff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 0x7f45f00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.280+0000 7f4603fff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 0x7f45f00a2460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4604101980 tx=0x7f45f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:33.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.282+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f45fc094d80 con 0x7f46041024c0 2023-08-25T04:44:33.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.485+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46041077c0 con 0x7f46041024c0 2023-08-25T04:44:33.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.487+0000 7f4601ffb700 1 -- 172.21.15.145:0/1718429743 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f45fc0983a0 con 0x7f46041024c0 2023-08-25T04:44:33.488 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:33.488 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:33.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 msgr2=0x7f45f00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 0x7f45f00a2460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4604101980 tx=0x7f45f4009260 comp rx=0 tx=0).stop 2023-08-25T04:44:33.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 msgr2=0x7f460419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007ee0 tx=0x7f45fc00d3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 shutdown_connections 2023-08-25T04:44:33.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f45f009ffa0 0x7f45f00a2460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46041024c0 0x7f460419db80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604100930 0x7f46041965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 --2- 172.21.15.145:0/1718429743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4604101b20 0x7f4604196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.489+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 >> 172.21.15.145:0/1718429743 conn(0x7f46040fbe60 msgr2=0x7f46040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:33.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.490+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 shutdown_connections 2023-08-25T04:44:33.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.490+0000 7f460a5ff700 1 -- 172.21.15.145:0/1718429743 wait complete. 2023-08-25T04:44:33.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:33.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:33.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 -- 172.21.15.145:0/637166596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 msgr2=0x7f5288104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 --2- 172.21.15.145:0/637166596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288104190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52780099c0 tx=0x7f5278009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 -- 172.21.15.145:0/637166596 shutdown_connections 2023-08-25T04:44:33.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 --2- 172.21.15.145:0/637166596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52881046d0 0x7f528810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 --2- 172.21.15.145:0/637166596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5288102b40 0x7f5288102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 --2- 172.21.15.145:0/637166596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288104190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 -- 172.21.15.145:0/637166596 >> 172.21.15.145:0/637166596 conn(0x7f52880fe0d0 msgr2=0x7f52881004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:33.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 -- 172.21.15.145:0/637166596 shutdown_connections 2023-08-25T04:44:33.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.713+0000 7f528f9ff700 1 -- 172.21.15.145:0/637166596 wait complete. 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 Processor -- start 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 -- start start 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5288102b40 0x7f5288198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52881046d0 0x7f528819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a03e0 con 0x7f5288103d30 2023-08-25T04:44:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a0520 con 0x7f52881046d0 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.714+0000 7f528f9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52881a06b0 con 0x7f5288102b40 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49862/0 (socket says 172.21.15.145:49862) 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 -- 172.21.15.145:0/659035355 learned_addr learned my addr 172.21.15.145:0/659035355 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528d79b700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5288102b40 0x7f5288198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528df9c700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52881046d0 0x7f528819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 -- 172.21.15.145:0/659035355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5288102b40 msgr2=0x7f5288198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5288102b40 0x7f5288198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 -- 172.21.15.145:0/659035355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52881046d0 msgr2=0x7f528819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52881046d0 0x7f528819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 -- 172.21.15.145:0/659035355 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780095f0 con 0x7f5288103d30 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f528cf9a700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5278009f50 tx=0x7f5278005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.715+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527801d070 con 0x7f5288103d30 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.716+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5278003f90 con 0x7f5288103d30 2023-08-25T04:44:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.716+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278022ce0 con 0x7f5288103d30 2023-08-25T04:44:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.716+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52881a0930 con 0x7f5288103d30 2023-08-25T04:44:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.716+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52881a0d00 con 0x7f5288103d30 2023-08-25T04:44:33.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.717+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528810e3a0 con 0x7f5288103d30 2023-08-25T04:44:33.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.719+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5278022470 con 0x7f5288103d30 2023-08-25T04:44:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.720+0000 7f527e7fc700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 0x7f52740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.720+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5278022e40 con 0x7f5288103d30 2023-08-25T04:44:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.720+0000 7f528d79b700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 0x7f52740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.721+0000 7f528d79b700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 0x7f52740a2410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5288103b90 tx=0x7f5284007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:33.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.724+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5278096430 con 0x7f5288103d30 2023-08-25T04:44:33.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.929+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5288109a90 con 0x7f5288103d30 2023-08-25T04:44:33.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.929+0000 7f527e7fc700 1 -- 172.21.15.145:0/659035355 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5278027020 con 0x7f5288103d30 2023-08-25T04:44:33.930 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:33.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 msgr2=0x7f52740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 0x7f52740a2410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5288103b90 tx=0x7f5284007400 comp rx=0 tx=0).stop 2023-08-25T04:44:33.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 msgr2=0x7f5288198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:33.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5278009f50 tx=0x7f5278005740 comp rx=0 tx=0).stop 2023-08-25T04:44:33.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 shutdown_connections 2023-08-25T04:44:33.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f527409ff50 0x7f52740a2410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52881046d0 0x7f528819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5288102b40 0x7f5288198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 --2- 172.21.15.145:0/659035355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5288103d30 0x7f5288198da0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:33.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 >> 172.21.15.145:0/659035355 conn(0x7f52880fe0d0 msgr2=0x7f5288100360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:33.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 shutdown_connections 2023-08-25T04:44:33.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:33.932+0000 7f528f9ff700 1 -- 172.21.15.145:0/659035355 wait complete. 2023-08-25T04:44:33.952 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:34.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:33 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1718429743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:34.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 -- 172.21.15.145:0/1076657888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d41024a0 msgr2=0x7f96d4108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 --2- 172.21.15.145:0/1076657888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d41024a0 0x7f96d4108d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00b0d0 tx=0x7f96cc00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 -- 172.21.15.145:0/1076657888 shutdown_connections 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 --2- 172.21.15.145:0/1076657888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d41024a0 0x7f96d4108d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 --2- 172.21.15.145:0/1076657888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 --2- 172.21.15.145:0/1076657888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d4101b00 0x7f96d4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 -- 172.21.15.145:0/1076657888 >> 172.21.15.145:0/1076657888 conn(0x7f96d40fbe60 msgr2=0x7f96d40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 -- 172.21.15.145:0/1076657888 shutdown_connections 2023-08-25T04:44:34.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.150+0000 7f96dabff700 1 -- 172.21.15.145:0/1076657888 wait complete. 2023-08-25T04:44:34.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.151+0000 7f96dabff700 1 Processor -- start 2023-08-25T04:44:34.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.151+0000 7f96dabff700 1 -- start start 2023-08-25T04:44:34.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 0x7f96d4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d41024a0 0x7f96d4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40739f0 con 0x7f96d4101b00 2023-08-25T04:44:34.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4073b60 con 0x7f96d4100910 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96dabff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41a0790 con 0x7f96d41024a0 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 0x7f96d4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47552/0 (socket says 172.21.15.145:47552) 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 -- 172.21.15.145:0/3865493722 learned_addr learned my addr 172.21.15.145:0/3865493722 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:34.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d919c700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d41024a0 0x7f96d4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 -- 172.21.15.145:0/3865493722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d41024a0 msgr2=0x7f96d4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d41024a0 0x7f96d4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 -- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 msgr2=0x7f96d4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 0x7f96d4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d899b700 1 -- 172.21.15.145:0/3865493722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8009520 con 0x7f96d4100910 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.152+0000 7f96d3fff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 0x7f96d4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96d899b700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96c800ea50 tx=0x7f96c800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:34.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800cbd0 con 0x7f96d4100910 2023-08-25T04:44:34.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c800cd30 con 0x7f96d4100910 2023-08-25T04:44:34.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8018680 con 0x7f96d4100910 2023-08-25T04:44:34.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc009d00 con 0x7f96d4100910 2023-08-25T04:44:34.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.153+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d41a0da0 con 0x7f96d4100910 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.154+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d4107840 con 0x7f96d4100910 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.156+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f96c8010430 con 0x7f96d4100910 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.157+0000 7f96d1ffb700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 0x7f96c40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.158+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f96c8014070 con 0x7f96d4100910 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.158+0000 7f96d3fff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 0x7f96c40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.158+0000 7f96d3fff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 0x7f96c40a2410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96c0011490 tx=0x7f96c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:34.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.159+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f96c8095670 con 0x7f96d4100910 2023-08-25T04:44:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:33 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1718429743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:33 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1718429743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:34.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.364+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96d410c330 con 0x7f96d4100910 2023-08-25T04:44:34.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.365+0000 7f96d1ffb700 1 -- 172.21.15.145:0/3865493722 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96c8098c90 con 0x7f96d4100910 2023-08-25T04:44:34.366 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:34.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 msgr2=0x7f96c40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 0x7f96c40a2410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96c0011490 tx=0x7f96c0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 msgr2=0x7f96d40746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96c800ea50 tx=0x7f96c800ed60 comp rx=0 tx=0).stop 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 shutdown_connections 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f96c409ff50 0x7f96c40a2410 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96d41024a0 0x7f96d4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d4100910 0x7f96d40746d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 --2- 172.21.15.145:0/3865493722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96d4101b00 0x7f96d4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 >> 172.21.15.145:0/3865493722 conn(0x7f96d40fbe60 msgr2=0x7f96d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:34.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 shutdown_connections 2023-08-25T04:44:34.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.367+0000 7f96dabff700 1 -- 172.21.15.145:0/3865493722 wait complete. 2023-08-25T04:44:34.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:34.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 -- 172.21.15.145:0/3763931076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0101b00 msgr2=0x7f38d0101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 --2- 172.21.15.145:0/3763931076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0101b00 0x7f38d0101f60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38c00099c0 tx=0x7f38c0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 -- 172.21.15.145:0/3763931076 shutdown_connections 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 --2- 172.21.15.145:0/3763931076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d01024a0 0x7f38d0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 --2- 172.21.15.145:0/3763931076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d0100910 0x7f38d0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 --2- 172.21.15.145:0/3763931076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0101b00 0x7f38d0101f60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 -- 172.21.15.145:0/3763931076 >> 172.21.15.145:0/3763931076 conn(0x7f38d00fbe60 msgr2=0x7f38d00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 -- 172.21.15.145:0/3763931076 shutdown_connections 2023-08-25T04:44:34.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.587+0000 7f38d7fff700 1 -- 172.21.15.145:0/3763931076 wait complete. 2023-08-25T04:44:34.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.588+0000 7f38d7fff700 1 Processor -- start 2023-08-25T04:44:34.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 -- start start 2023-08-25T04:44:34.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d0101b00 0x7f38d0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d01024a0 0x7f38d019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01a03e0 con 0x7f38d0100910 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01a0520 con 0x7f38d01024a0 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d7fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01a06b0 con 0x7f38d0101b00 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d5d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49892/0 (socket says 172.21.15.145:49892) 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d5d9b700 1 -- 172.21.15.145:0/668281755 learned_addr learned my addr 172.21.15.145:0/668281755 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d559a700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d0101b00 0x7f38d0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.589+0000 7f38d659c700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d01024a0 0x7f38d019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 -- 172.21.15.145:0/668281755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d0101b00 msgr2=0x7f38d0198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d0101b00 0x7f38d0198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 -- 172.21.15.145:0/668281755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d01024a0 msgr2=0x7f38d019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d01024a0 0x7f38d019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 -- 172.21.15.145:0/668281755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00095f0 con 0x7f38d0100910 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d5d9b700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e970 tx=0x7f38cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc009790 con 0x7f38d0100910 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d01a0990 con 0x7f38d0100910 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38cc00c7e0 con 0x7f38d0100910 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003c70 con 0x7f38d0100910 2023-08-25T04:44:34.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.590+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d01a0ef0 con 0x7f38d0100910 2023-08-25T04:44:34.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.591+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0192f60 con 0x7f38d0100910 2023-08-25T04:44:34.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.593+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f38cc00cc50 con 0x7f38d0100910 2023-08-25T04:44:34.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.594+0000 7f38c6ffd700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 0x7f38bc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:34.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.595+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f38cc014070 con 0x7f38d0100910 2023-08-25T04:44:34.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.595+0000 7f38d559a700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 0x7f38bc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:34.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.595+0000 7f38d559a700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 0x7f38bc0a2410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b580 tx=0x7f38c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:34.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.598+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f38cc095160 con 0x7f38d0100910 2023-08-25T04:44:34.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.799+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38d01a0b20 con 0x7f38d0100910 2023-08-25T04:44:34.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.799+0000 7f38c6ffd700 1 -- 172.21.15.145:0/668281755 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38d01a0b20 con 0x7f38d0100910 2023-08-25T04:44:34.800 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:34.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 msgr2=0x7f38bc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 0x7f38bc0a2410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b580 tx=0x7f38c0005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 msgr2=0x7f38d0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:34.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e970 tx=0x7f38cc00ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:34.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 shutdown_connections 2023-08-25T04:44:34.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f38bc09ff50 0x7f38bc0a2410 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d01024a0 0x7f38d019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38d0100910 0x7f38d0198860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 --2- 172.21.15.145:0/668281755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d0101b00 0x7f38d0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:34.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.801+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 >> 172.21.15.145:0/668281755 conn(0x7f38d00fbe60 msgr2=0x7f38d0107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:34.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.802+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 shutdown_connections 2023-08-25T04:44:34.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:34.802+0000 7f38d7fff700 1 -- 172.21.15.145:0/668281755 wait complete. 2023-08-25T04:44:34.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:35.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 -- 172.21.15.145:0/864781471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c1024c0 msgr2=0x7fa94c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 --2- 172.21.15.145:0/864781471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c1024c0 0x7fa94c108d70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b0d0 tx=0x7fa94400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 -- 172.21.15.145:0/864781471 shutdown_connections 2023-08-25T04:44:35.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 --2- 172.21.15.145:0/864781471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c1024c0 0x7fa94c108d70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 --2- 172.21.15.145:0/864781471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c100930 0x7fa94c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 --2- 172.21.15.145:0/864781471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.021+0000 7fa9533ff700 1 -- 172.21.15.145:0/864781471 >> 172.21.15.145:0/864781471 conn(0x7fa94c0fbea0 msgr2=0x7fa94c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:35.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.022+0000 7fa9533ff700 1 -- 172.21.15.145:0/864781471 shutdown_connections 2023-08-25T04:44:35.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.022+0000 7fa9533ff700 1 -- 172.21.15.145:0/864781471 wait complete. 2023-08-25T04:44:35.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 Processor -- start 2023-08-25T04:44:35.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 -- start start 2023-08-25T04:44:35.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 0x7fa94c19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1a0450 con 0x7fa94c100930 2023-08-25T04:44:35.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1a0590 con 0x7fa94c101b20 2023-08-25T04:44:35.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa9533ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1a0720 con 0x7fa94c1024c0 2023-08-25T04:44:35.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa95119b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa95119b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49912/0 (socket says 172.21.15.145:49912) 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa95119b700 1 -- 172.21.15.145:0/3198929988 learned_addr learned my addr 172.21.15.145:0/3198929988 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa95199c700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 0x7fa94c19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.023+0000 7fa95099a700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 -- 172.21.15.145:0/3198929988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 msgr2=0x7fa94c19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 0x7fa94c19fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 -- 172.21.15.145:0/3198929988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 msgr2=0x7fa94c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 -- 172.21.15.145:0/3198929988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944009d00 con 0x7fa94c100930 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95199c700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 0x7fa94c19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95099a700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa95119b700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e970 tx=0x7fa94800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948009790 con 0x7fa94c100930 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c1a0a00 con 0x7fa94c100930 2023-08-25T04:44:35.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9480044e0 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948003e30 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.024+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c1a0f10 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.025+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c10c150 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.027+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa948004000 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.028+0000 7fa9427fc700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 0x7fa9380a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.029+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa948014070 con 0x7fa94c100930 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.029+0000 7fa95099a700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 0x7fa9380a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.029+0000 7fa95099a700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 0x7fa9380a2410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009cb0 tx=0x7fa93c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.032+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa9480952b0 con 0x7fa94c100930 2023-08-25T04:44:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:34 smithi145 ceph-mon[97681]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:34 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/659035355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:34 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3865493722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:34 smithi162 ceph-mon[130941]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:34 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/659035355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:34 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3865493722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:34 smithi183 ceph-mon[131414]: pgmap v319: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-08-25T04:44:35.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:34 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/659035355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:34 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3865493722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:35.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.234+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa94c1a1160 con 0x7fa94c100930 2023-08-25T04:44:35.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.235+0000 7fa9427fc700 1 -- 172.21.15.145:0/3198929988 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa9480988d0 con 0x7fa94c100930 2023-08-25T04:44:35.236 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:35.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 msgr2=0x7fa9380a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 0x7fa9380a2410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009cb0 tx=0x7fa93c009260 comp rx=0 tx=0).stop 2023-08-25T04:44:35.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 msgr2=0x7fa94c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e970 tx=0x7fa94800ed30 comp rx=0 tx=0).stop 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 shutdown_connections 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa93809ff50 0x7fa9380a2410 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa94c1024c0 0x7fa94c19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa94c100930 0x7fa94c1988d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 --2- 172.21.15.145:0/3198929988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c101b20 0x7fa94c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 >> 172.21.15.145:0/3198929988 conn(0x7fa94c0fbea0 msgr2=0x7fa94c107350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:35.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 shutdown_connections 2023-08-25T04:44:35.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.237+0000 7fa9533ff700 1 -- 172.21.15.145:0/3198929988 wait complete. 2023-08-25T04:44:35.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:35.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 -- 172.21.15.145:0/2674065278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 msgr2=0x7fb608100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/2674065278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608100d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6040099c0 tx=0x7fb604009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 -- 172.21.15.145:0/2674065278 shutdown_connections 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/2674065278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6081024b0 0x7fb608108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/2674065278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608100d40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/2674065278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb608101b10 0x7fb608101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 -- 172.21.15.145:0/2674065278 >> 172.21.15.145:0/2674065278 conn(0x7fb6080fbe50 msgr2=0x7fb6080fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:35.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 -- 172.21.15.145:0/2674065278 shutdown_connections 2023-08-25T04:44:35.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.453+0000 7fb60f5ff700 1 -- 172.21.15.145:0/2674065278 wait complete. 2023-08-25T04:44:35.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 Processor -- start 2023-08-25T04:44:35.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 -- start start 2023-08-25T04:44:35.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb608101b10 0x7fb608196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6081024b0 0x7fb60819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60819e1b0 con 0x7fb608100920 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60819e2f0 con 0x7fb608101b10 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.454+0000 7fb60f5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60819e480 con 0x7fb6081024b0 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49918/0 (socket says 172.21.15.145:49918) 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 -- 172.21.15.145:0/1951844718 learned_addr learned my addr 172.21.15.145:0/1951844718 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60cb9a700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb608101b10 0x7fb608196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60db9c700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6081024b0 0x7fb60819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 -- 172.21.15.145:0/1951844718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6081024b0 msgr2=0x7fb60819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6081024b0 0x7fb60819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 -- 172.21.15.145:0/1951844718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb608101b10 msgr2=0x7fb608196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb608101b10 0x7fb608196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 -- 172.21.15.145:0/1951844718 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040095f0 con 0x7fb608100920 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.455+0000 7fb60d39b700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb604009f50 tx=0x7fb604005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.456+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60401d070 con 0x7fb608100920 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.456+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb604003f90 con 0x7fb608100920 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.456+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60819e700 con 0x7fb608100920 2023-08-25T04:44:35.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.456+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604022d00 con 0x7fb608100920 2023-08-25T04:44:35.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.456+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60819eb80 con 0x7fb608100920 2023-08-25T04:44:35.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.457+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60810c1a0 con 0x7fb608100920 2023-08-25T04:44:35.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.459+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fb604022470 con 0x7fb608100920 2023-08-25T04:44:35.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.462+0000 7fb5fe7fc700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 0x7fb5f40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.462+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb6040ccd90 con 0x7fb608100920 2023-08-25T04:44:35.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.462+0000 7fb60cb9a700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 0x7fb5f40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.462+0000 7fb60cb9a700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 0x7fb5f40a24e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8004630 tx=0x7fb5f800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.464+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fb604096850 con 0x7fb608100920 2023-08-25T04:44:35.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.709+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb608107840 con 0x7fb608100920 2023-08-25T04:44:35.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.710+0000 7fb5fe7fc700 1 -- 172.21.15.145:0/1951844718 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7fb604022e60 con 0x7fb608100920 2023-08-25T04:44:35.711 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:35.711 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:35.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 msgr2=0x7fb5f40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 0x7fb5f40a24e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8004630 tx=0x7fb5f800b3c0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 msgr2=0x7fb608196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb604009f50 tx=0x7fb604005740 comp rx=0 tx=0).stop 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 shutdown_connections 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fb5f40a0020 0x7fb5f40a24e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6081024b0 0x7fb60819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb608100920 0x7fb608196630 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 --2- 172.21.15.145:0/1951844718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb608101b10 0x7fb608196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 >> 172.21.15.145:0/1951844718 conn(0x7fb6080fbe50 msgr2=0x7fb6080fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:35.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 shutdown_connections 2023-08-25T04:44:35.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.713+0000 7fb60f5ff700 1 -- 172.21.15.145:0/1951844718 wait complete. 2023-08-25T04:44:35.715 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:35.732 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:35.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 -- 172.21.15.145:0/1502638953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f0100930 msgr2=0x7f25f0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 --2- 172.21.15.145:0/1502638953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f0100930 0x7f25f0100d50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f25e00099c0 tx=0x7f25e0009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 -- 172.21.15.145:0/1502638953 shutdown_connections 2023-08-25T04:44:35.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 --2- 172.21.15.145:0/1502638953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f01024c0 0x7f25f0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 --2- 172.21.15.145:0/1502638953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f0100930 0x7f25f0100d50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 --2- 172.21.15.145:0/1502638953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 0x7f25f0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 -- 172.21.15.145:0/1502638953 >> 172.21.15.145:0/1502638953 conn(0x7f25f00fbe60 msgr2=0x7f25f00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:35.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 -- 172.21.15.145:0/1502638953 shutdown_connections 2023-08-25T04:44:35.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.929+0000 7f25f61ff700 1 -- 172.21.15.145:0/1502638953 wait complete. 2023-08-25T04:44:35.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.930+0000 7f25f61ff700 1 Processor -- start 2023-08-25T04:44:35.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.930+0000 7f25f61ff700 1 -- start start 2023-08-25T04:44:35.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 0x7f25f0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 0x7f25f0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 0x7f25f019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e1a0 con 0x7f25f01024c0 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e2e0 con 0x7f25f0100930 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25f61ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e470 con 0x7f25f0101b20 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 0x7f25f0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 0x7f25f019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 0x7f25f0196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47628/0 (socket says 172.21.15.145:47628) 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 0x7f25f0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25ef7fe700 1 -- 172.21.15.145:0/4107521713 learned_addr learned my addr 172.21.15.145:0/4107521713 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 -- 172.21.15.145:0/4107521713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 msgr2=0x7f25f0196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 0x7f25f0196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 -- 172.21.15.145:0/4107521713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 msgr2=0x7f25f0196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 0x7f25f0196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.931+0000 7f25effff700 1 -- 172.21.15.145:0/4107521713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00095f0 con 0x7f25f01024c0 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25effff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 0x7f25f019dbd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007f10 tx=0x7f25e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8009ea0 con 0x7f25f01024c0 2023-08-25T04:44:35.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e800f040 con 0x7f25f01024c0 2023-08-25T04:44:35.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f019e750 con 0x7f25f01024c0 2023-08-25T04:44:35.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8004350 con 0x7f25f01024c0 2023-08-25T04:44:35.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.932+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f019ec60 con 0x7f25f01024c0 2023-08-25T04:44:35.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.933+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f0107ec0 con 0x7f25f01024c0 2023-08-25T04:44:35.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.935+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f25e800db20 con 0x7f25f01024c0 2023-08-25T04:44:35.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.936+0000 7f25ecff9700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 0x7f25d80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:35.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.936+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f25e80cada0 con 0x7f25f01024c0 2023-08-25T04:44:35.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.936+0000 7f25ef7fe700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 0x7f25d80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:35.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.937+0000 7f25ef7fe700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 0x7f25d80a24e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:35.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:35.940+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f25e80947d0 con 0x7f25f01024c0 2023-08-25T04:44:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:35 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/668281755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:35 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3198929988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:35 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1951844718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:36.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.142+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25f010c190 con 0x7f25f01024c0 2023-08-25T04:44:36.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.144+0000 7f25ecff9700 1 -- 172.21.15.145:0/4107521713 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f25e8019090 con 0x7f25f01024c0 2023-08-25T04:44:36.145 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:36.145 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:36.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 msgr2=0x7f25d80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 0x7f25d80a24e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000c00 tx=0x7f25e0005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 msgr2=0x7f25f019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 0x7f25f019dbd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007f10 tx=0x7f25e800d3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 shutdown_connections 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f25d80a0020 0x7f25d80a24e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f01024c0 0x7f25f019dbd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25f0100930 0x7f25f0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 --2- 172.21.15.145:0/4107521713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25f0101b20 0x7f25f0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 >> 172.21.15.145:0/4107521713 conn(0x7f25f00fbe60 msgr2=0x7f25f0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:36.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 shutdown_connections 2023-08-25T04:44:36.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.147+0000 7f25f61ff700 1 -- 172.21.15.145:0/4107521713 wait complete. 2023-08-25T04:44:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:35 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/668281755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:35 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3198929988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:35 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1951844718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:36.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:36.169 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:35 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/668281755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:35 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3198929988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:36.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:35 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1951844718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:36.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 -- 172.21.15.145:0/1046997698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc1024c0 msgr2=0x7f58dc108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 --2- 172.21.15.145:0/1046997698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc1024c0 0x7f58dc108d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b0d0 tx=0x7f58d400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 -- 172.21.15.145:0/1046997698 shutdown_connections 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 --2- 172.21.15.145:0/1046997698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc1024c0 0x7f58dc108d70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 --2- 172.21.15.145:0/1046997698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc100930 0x7f58dc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 --2- 172.21.15.145:0/1046997698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc101b20 0x7f58dc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 -- 172.21.15.145:0/1046997698 >> 172.21.15.145:0/1046997698 conn(0x7f58dc0fbea0 msgr2=0x7f58dc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:36.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 -- 172.21.15.145:0/1046997698 shutdown_connections 2023-08-25T04:44:36.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.368+0000 7f58e25ff700 1 -- 172.21.15.145:0/1046997698 wait complete. 2023-08-25T04:44:36.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 Processor -- start 2023-08-25T04:44:36.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 -- start start 2023-08-25T04:44:36.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 0x7f58dc196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc101b20 0x7f58dc196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 0x7f58dc19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19e2a0 con 0x7f58dc100930 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19e3e0 con 0x7f58dc1024c0 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58e25ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19e570 con 0x7f58dc101b20 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.369+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 0x7f58dc196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58e0b9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 0x7f58dc19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc101b20 0x7f58dc196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58e0b9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 0x7f58dc19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47648/0 (socket says 172.21.15.145:47648) 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58e0b9c700 1 -- 172.21.15.145:0/909539758 learned_addr learned my addr 172.21.15.145:0/909539758 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 -- 172.21.15.145:0/909539758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc101b20 msgr2=0x7f58dc196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc101b20 0x7f58dc196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 -- 172.21.15.145:0/909539758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 msgr2=0x7f58dc19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 0x7f58dc19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 -- 172.21.15.145:0/909539758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009d00 con 0x7f58dc100930 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58dbfff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 0x7f58dc196720 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e9c0 tx=0x7f58cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc004220 con 0x7f58dc100930 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc19e850 con 0x7f58dc100930 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58cc004380 con 0x7f58dc100930 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.370+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003be0 con 0x7f58dc100930 2023-08-25T04:44:36.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.371+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc19edb0 con 0x7f58dc100930 2023-08-25T04:44:36.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.372+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc107e50 con 0x7f58dc100930 2023-08-25T04:44:36.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.373+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f58cc003d40 con 0x7f58dc100930 2023-08-25T04:44:36.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.375+0000 7f58d97fa700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 0x7f58c40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.375+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f58cc014070 con 0x7f58dc100930 2023-08-25T04:44:36.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.375+0000 7f58db7fe700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 0x7f58c40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.376+0000 7f58db7fe700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 0x7f58c40a2410 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009c40 tx=0x7f58d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:36.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.379+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f58cc095240 con 0x7f58dc100930 2023-08-25T04:44:36.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.579+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58dc10c1a0 con 0x7f58dc100930 2023-08-25T04:44:36.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.579+0000 7f58d97fa700 1 -- 172.21.15.145:0/909539758 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58cc098860 con 0x7f58dc100930 2023-08-25T04:44:36.581 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:36.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 msgr2=0x7f58c40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 0x7f58c40a2410 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009c40 tx=0x7f58d0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:36.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 msgr2=0x7f58dc196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 0x7f58dc196720 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e9c0 tx=0x7f58cc00ecd0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 shutdown_connections 2023-08-25T04:44:36.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f58c409ff50 0x7f58c40a2410 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58dc1024c0 0x7f58dc19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58dc100930 0x7f58dc196720 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 --2- 172.21.15.145:0/909539758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc101b20 0x7f58dc196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 >> 172.21.15.145:0/909539758 conn(0x7f58dc0fbea0 msgr2=0x7f58dc104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:36.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 shutdown_connections 2023-08-25T04:44:36.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.582+0000 7f58e25ff700 1 -- 172.21.15.145:0/909539758 wait complete. 2023-08-25T04:44:36.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:36.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/515000836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf441024c0 msgr2=0x7fdf44108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/515000836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf441024c0 0x7fdf44108d70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00b0d0 tx=0x7fdf3c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/515000836 shutdown_connections 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/515000836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf441024c0 0x7fdf44108d70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/515000836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf44100930 0x7fdf44100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/515000836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/515000836 >> 172.21.15.145:0/515000836 conn(0x7fdf440fbea0 msgr2=0x7fdf440fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:36.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/515000836 shutdown_connections 2023-08-25T04:44:36.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.802+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/515000836 wait complete. 2023-08-25T04:44:36.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 Processor -- start 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 -- start start 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 0x7fdf4419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4419e2a0 con 0x7fdf44100930 2023-08-25T04:44:36.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4419e3e0 con 0x7fdf441024c0 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf4a3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4419e570 con 0x7fdf44101b20 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.803+0000 7fdf43fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49958/0 (socket says 172.21.15.145:49958) 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 -- 172.21.15.145:0/81640793 learned_addr learned my addr 172.21.15.145:0/81640793 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf3bfff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf4899c700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 0x7fdf4419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 -- 172.21.15.145:0/81640793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 msgr2=0x7fdf44196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 -- 172.21.15.145:0/81640793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 msgr2=0x7fdf4419dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 0x7fdf4419dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 -- 172.21.15.145:0/81640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c009d00 con 0x7fdf44100930 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf4899c700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 0x7fdf4419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf3bfff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:36.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf43fff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e970 tx=0x7fdf3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:36.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34009790 con 0x7fdf44100930 2023-08-25T04:44:36.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3400eec0 con 0x7fdf44100930 2023-08-25T04:44:36.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4419e850 con 0x7fdf44100930 2023-08-25T04:44:36.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.804+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34003cf0 con 0x7fdf44100930 2023-08-25T04:44:36.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.805+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4419ed60 con 0x7fdf44100930 2023-08-25T04:44:36.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.806+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf44107ec0 con 0x7fdf44100930 2023-08-25T04:44:36.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.808+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fdf3400ccd0 con 0x7fdf44100930 2023-08-25T04:44:36.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.810+0000 7fdf41ffb700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 0x7fdf300a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:36.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.810+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdf34014070 con 0x7fdf44100930 2023-08-25T04:44:36.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.810+0000 7fdf3bfff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 0x7fdf300a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:36.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.811+0000 7fdf3bfff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 0x7fdf300a2410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011490 tx=0x7fdf2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:36.813 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:36.812+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fdf34095360 con 0x7fdf44100930 2023-08-25T04:44:37.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.014+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf4410c350 con 0x7fdf44100930 2023-08-25T04:44:37.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.014+0000 7fdf41ffb700 1 -- 172.21.15.145:0/81640793 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf34098980 con 0x7fdf44100930 2023-08-25T04:44:37.015 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:37.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 msgr2=0x7fdf300a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 0x7fdf300a2410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011490 tx=0x7fdf2c009260 comp rx=0 tx=0).stop 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 msgr2=0x7fdf44196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e970 tx=0x7fdf3400ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 shutdown_connections 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fdf3009ff50 0x7fdf300a2410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf441024c0 0x7fdf4419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf44100930 0x7fdf44196720 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 --2- 172.21.15.145:0/81640793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf44101b20 0x7fdf44196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 >> 172.21.15.145:0/81640793 conn(0x7fdf440fbea0 msgr2=0x7fdf44105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:37.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 shutdown_connections 2023-08-25T04:44:37.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.017+0000 7fdf4a3ff700 1 -- 172.21.15.145:0/81640793 wait complete. 2023-08-25T04:44:37.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:36 smithi145 ceph-mon[97681]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:37.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:36 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4107521713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:37.034 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:36 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/909539758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:37.036 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:37.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:36 smithi162 ceph-mon[130941]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:36 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4107521713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:37.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:36 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/909539758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:36 smithi183 ceph-mon[131414]: pgmap v320: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-08-25T04:44:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:36 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4107521713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:37.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:36 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/909539758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:37.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.235+0000 7f5a93bff700 1 -- 172.21.15.145:0/1146822519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c101b20 msgr2=0x7f5a8c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.235+0000 7f5a93bff700 1 --2- 172.21.15.145:0/1146822519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c101b20 0x7f5a8c101f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009990 tx=0x7f5a7c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 -- 172.21.15.145:0/1146822519 shutdown_connections 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 --2- 172.21.15.145:0/1146822519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c1024c0 0x7f5a8c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 --2- 172.21.15.145:0/1146822519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 --2- 172.21.15.145:0/1146822519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c101b20 0x7f5a8c101f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 -- 172.21.15.145:0/1146822519 >> 172.21.15.145:0/1146822519 conn(0x7f5a8c0fbea0 msgr2=0x7f5a8c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:37.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 -- 172.21.15.145:0/1146822519 shutdown_connections 2023-08-25T04:44:37.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.236+0000 7f5a93bff700 1 -- 172.21.15.145:0/1146822519 wait complete. 2023-08-25T04:44:37.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 Processor -- start 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 -- start start 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c101b20 0x7f5a8c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c1024c0 0x7f5a8c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c1a0350 con 0x7f5a8c100930 2023-08-25T04:44:37.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c1a0490 con 0x7f5a8c101b20 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a93bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c1a0620 con 0x7f5a8c1024c0 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a9199b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.237+0000 7f5a9199b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49972/0 (socket says 172.21.15.145:49972) 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 -- 172.21.15.145:0/363068008 learned_addr learned my addr 172.21.15.145:0/363068008 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9219c700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c1024c0 0x7f5a8c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9119a700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c101b20 0x7f5a8c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 -- 172.21.15.145:0/363068008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c1024c0 msgr2=0x7f5a8c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c1024c0 0x7f5a8c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 -- 172.21.15.145:0/363068008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c101b20 msgr2=0x7f5a8c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c101b20 0x7f5a8c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 -- 172.21.15.145:0/363068008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0095f0 con 0x7f5a8c100930 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a9199b700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e970 tx=0x7f5a8800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88009790 con 0x7f5a8c100930 2023-08-25T04:44:37.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c1a0900 con 0x7f5a8c100930 2023-08-25T04:44:37.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a8800c7e0 con 0x7f5a8c100930 2023-08-25T04:44:37.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88003c70 con 0x7f5a8c100930 2023-08-25T04:44:37.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.238+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c1a0e60 con 0x7f5a8c100930 2023-08-25T04:44:37.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.239+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c192f30 con 0x7f5a8c100930 2023-08-25T04:44:37.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.241+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f5a8800cc50 con 0x7f5a8c100930 2023-08-25T04:44:37.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.244+0000 7f5a82ffd700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 0x7f5a780a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.244+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5a88014070 con 0x7f5a8c100930 2023-08-25T04:44:37.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.244+0000 7f5a9119a700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 0x7f5a780a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.244+0000 7f5a9119a700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 0x7f5a780a2410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b680 tx=0x7f5a7c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:37.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.246+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5a880952c0 con 0x7f5a8c100930 2023-08-25T04:44:37.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.447+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a8c1a0a90 con 0x7f5a8c100930 2023-08-25T04:44:37.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.448+0000 7f5a82ffd700 1 -- 172.21.15.145:0/363068008 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a8c1a0a90 con 0x7f5a8c100930 2023-08-25T04:44:37.449 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:37.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 msgr2=0x7f5a780a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 0x7f5a780a2410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b680 tx=0x7f5a7c005fd0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 msgr2=0x7f5a8c1987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e970 tx=0x7f5a8800ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:37.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 shutdown_connections 2023-08-25T04:44:37.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f5a7809ff50 0x7f5a780a2410 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a8c1024c0 0x7f5a8c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a8c100930 0x7f5a8c1987d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 --2- 172.21.15.145:0/363068008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a8c101b20 0x7f5a8c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 >> 172.21.15.145:0/363068008 conn(0x7f5a8c0fbea0 msgr2=0x7f5a8c107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:37.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 shutdown_connections 2023-08-25T04:44:37.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.450+0000 7f5a93bff700 1 -- 172.21.15.145:0/363068008 wait complete. 2023-08-25T04:44:37.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:37.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 -- 172.21.15.145:0/3553162676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc100520 msgr2=0x7fbabc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 --2- 172.21.15.145:0/3553162676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc100520 0x7fbabc1009a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009f90 tx=0x7fbab400b330 comp rx=0 tx=0).stop 2023-08-25T04:44:37.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 -- 172.21.15.145:0/3553162676 shutdown_connections 2023-08-25T04:44:37.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 --2- 172.21.15.145:0/3553162676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc100520 0x7fbabc1009a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 --2- 172.21.15.145:0/3553162676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc0ff200 0x7fbabc0ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 --2- 172.21.15.145:0/3553162676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 0x7fbabc0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 -- 172.21.15.145:0/3553162676 >> 172.21.15.145:0/3553162676 conn(0x7fbabc0fad80 msgr2=0x7fbabc0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:37.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 -- 172.21.15.145:0/3553162676 shutdown_connections 2023-08-25T04:44:37.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.670+0000 7fbac0dff700 1 -- 172.21.15.145:0/3553162676 wait complete. 2023-08-25T04:44:37.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.671+0000 7fbac0dff700 1 Processor -- start 2023-08-25T04:44:37.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.671+0000 7fbac0dff700 1 -- start start 2023-08-25T04:44:37.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 0x7fbabc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 0x7fbabc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a01d0 con 0x7fbabc100520 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a0310 con 0x7fbabc0ffb60 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbac0dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a04a0 con 0x7fbabc0ff200 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 0x7fbabc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 0x7fbabc198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41670/0 (socket says 172.21.15.145:41670) 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbab9d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 0x7fbabc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbaba59c700 1 -- 172.21.15.145:0/212892547 learned_addr learned my addr 172.21.15.145:0/212892547 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49988/0 (socket says 172.21.15.145:49988) 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 -- 172.21.15.145:0/212892547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 msgr2=0x7fbabc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 0x7fbabc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 -- 172.21.15.145:0/212892547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 msgr2=0x7fbabc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 0x7fbabc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.672+0000 7fbabad9d700 1 -- 172.21.15.145:0/212892547 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4009c40 con 0x7fbabc100520 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbabad9d700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbab400b6c0 tx=0x7fbab4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400e040 con 0x7fbabc100520 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc1a0720 con 0x7fbabc100520 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab400be00 con 0x7fbabc100520 2023-08-25T04:44:37.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab401b6b0 con 0x7fbabc100520 2023-08-25T04:44:37.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.673+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc1a0bf0 con 0x7fbabc100520 2023-08-25T04:44:37.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.674+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc109fe0 con 0x7fbabc100520 2023-08-25T04:44:37.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.676+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fbab401b810 con 0x7fbabc100520 2023-08-25T04:44:37.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.677+0000 7fbaab7fe700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 0x7fbaa40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:37.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.677+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbab40cc9b0 con 0x7fbabc100520 2023-08-25T04:44:37.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.677+0000 7fbaba59c700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 0x7fbaa40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:37.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.678+0000 7fbaba59c700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 0x7fbaa40a2490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005950 tx=0x7fbaac009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:37.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.681+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbab4096470 con 0x7fbabc100520 2023-08-25T04:44:37.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.885+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbabc02cde0 con 0x7fbabc100520 2023-08-25T04:44:37.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.886+0000 7fbaab7fe700 1 -- 172.21.15.145:0/212892547 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbab4099a90 con 0x7fbabc100520 2023-08-25T04:44:37.887 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:37.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 msgr2=0x7fbaa40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 0x7fbaa40a2490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005950 tx=0x7fbaac009310 comp rx=0 tx=0).stop 2023-08-25T04:44:37.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 msgr2=0x7fbabc19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:37.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbab400b6c0 tx=0x7fbab4004d00 comp rx=0 tx=0).stop 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 shutdown_connections 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbaa409ffd0 0x7fbaa40a2490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbabc100520 0x7fbabc19fc00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc0ff200 0x7fbabc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 --2- 172.21.15.145:0/212892547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbabc0ffb60 0x7fbabc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.888+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 >> 172.21.15.145:0/212892547 conn(0x7fbabc0fad80 msgr2=0x7fbabc0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.889+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 shutdown_connections 2023-08-25T04:44:37.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:37.889+0000 7fbac0dff700 1 -- 172.21.15.145:0/212892547 wait complete. 2023-08-25T04:44:37.908 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:37 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/81640793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:37 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/363068008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3800439758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8101b20 msgr2=0x7f27e8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3800439758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8101b20 0x7f27e8101f80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0099c0 tx=0x7f27dc009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3800439758 shutdown_connections 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3800439758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 0x7f27e8108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3800439758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8100930 0x7f27e8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3800439758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8101b20 0x7f27e8101f80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3800439758 >> 172.21.15.145:0/3800439758 conn(0x7f27e80fbe60 msgr2=0x7f27e80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3800439758 shutdown_connections 2023-08-25T04:44:38.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.105+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3800439758 wait complete. 2023-08-25T04:44:38.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.106+0000 7f27ed1ff700 1 Processor -- start 2023-08-25T04:44:38.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.106+0000 7f27ed1ff700 1 -- start start 2023-08-25T04:44:38.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8101b20 0x7f27e8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 0x7f27e819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81a0310 con 0x7f27e8100930 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81a0450 con 0x7f27e81024c0 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27ed1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81a05e0 con 0x7f27e8101b20 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50016/0 (socket says 172.21.15.145:50016) 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 -- 172.21.15.145:0/3204741978 learned_addr learned my addr 172.21.15.145:0/3204741978 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:38.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e659c700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8101b20 0x7f27e8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e759e700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 0x7f27e819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 -- 172.21.15.145:0/3204741978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8101b20 msgr2=0x7f27e8198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8101b20 0x7f27e8198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 -- 172.21.15.145:0/3204741978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 msgr2=0x7f27e819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 0x7f27e819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.107+0000 7f27e6d9d700 1 -- 172.21.15.145:0/3204741978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0095f0 con 0x7f27e8100930 2023-08-25T04:44:38.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27e6d9d700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e970 tx=0x7f27d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:38.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009790 con 0x7f27e8100930 2023-08-25T04:44:38.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e81a08c0 con 0x7f27e8100930 2023-08-25T04:44:38.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d800eec0 con 0x7f27e8100930 2023-08-25T04:44:38.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8003cf0 con 0x7f27e8100930 2023-08-25T04:44:38.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.108+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a0e20 con 0x7f27e8100930 2023-08-25T04:44:38.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.109+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e8192f30 con 0x7f27e8100930 2023-08-25T04:44:38.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.111+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f27d800ccd0 con 0x7f27e8100930 2023-08-25T04:44:38.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.112+0000 7f27cffff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 0x7f27d00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.113+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f27d8014070 con 0x7f27e8100930 2023-08-25T04:44:38.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.113+0000 7f27e659c700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 0x7f27d00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.113+0000 7f27e659c700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 0x7f27d00a2460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b580 tx=0x7f27dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:38.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.116+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f27d80951f0 con 0x7f27e8100930 2023-08-25T04:44:38.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:37 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/81640793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:37 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/363068008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:37 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/81640793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:37 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/363068008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:38.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.364+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27e810c140 con 0x7f27e8100930 2023-08-25T04:44:38.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.365+0000 7f27cffff700 1 -- 172.21.15.145:0/3204741978 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f27d8098810 con 0x7f27e8100930 2023-08-25T04:44:38.366 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:38.367 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 msgr2=0x7f27d00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 0x7f27d00a2460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b580 tx=0x7f27dc005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 msgr2=0x7f27e8198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e970 tx=0x7f27d800ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 shutdown_connections 2023-08-25T04:44:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f27d009ffa0 0x7f27d00a2460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27e81024c0 0x7f27e819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27e8100930 0x7f27e8198790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 --2- 172.21.15.145:0/3204741978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e8101b20 0x7f27e8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 >> 172.21.15.145:0/3204741978 conn(0x7f27e80fbe60 msgr2=0x7f27e81073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 shutdown_connections 2023-08-25T04:44:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.368+0000 7f27ed1ff700 1 -- 172.21.15.145:0/3204741978 wait complete. 2023-08-25T04:44:38.372 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:38.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 -- 172.21.15.145:0/2508862216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50074960 msgr2=0x7fbb50072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 --2- 172.21.15.145:0/2508862216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50074960 0x7fbb50072db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009990 tx=0x7fbb44009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 -- 172.21.15.145:0/2508862216 shutdown_connections 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 --2- 172.21.15.145:0/2508862216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb501087e0 0x7fbb5010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 --2- 172.21.15.145:0/2508862216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50074960 0x7fbb50072db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.585 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 --2- 172.21.15.145:0/2508862216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50073380 0x7fbb50073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 -- 172.21.15.145:0/2508862216 >> 172.21.15.145:0/2508862216 conn(0x7fbb500fbe50 msgr2=0x7fbb500fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:38.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 -- 172.21.15.145:0/2508862216 shutdown_connections 2023-08-25T04:44:38.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.584+0000 7fbb56fff700 1 -- 172.21.15.145:0/2508862216 wait complete. 2023-08-25T04:44:38.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 Processor -- start 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 -- start start 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb50074960 0x7fbb50198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb501087e0 0x7fbb5019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501a03a0 con 0x7fbb50073380 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501a04e0 con 0x7fbb501087e0 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.585+0000 7fbb56fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501a0670 con 0x7fbb50074960 2023-08-25T04:44:38.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50022/0 (socket says 172.21.15.145:50022) 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 -- 172.21.15.145:0/1186300910 learned_addr learned my addr 172.21.15.145:0/1186300910 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb4ffff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb50074960 0x7fbb50198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb5559c700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb501087e0 0x7fbb5019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 -- 172.21.15.145:0/1186300910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb50074960 msgr2=0x7fbb50198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb50074960 0x7fbb50198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 -- 172.21.15.145:0/1186300910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb501087e0 msgr2=0x7fbb5019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb501087e0 0x7fbb5019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 -- 172.21.15.145:0/1186300910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440095f0 con 0x7fbb50073380 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.586+0000 7fbb54d9b700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb44004740 tx=0x7fbb44009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.587+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401d070 con 0x7fbb50073380 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.587+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb44004830 con 0x7fbb50073380 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.587+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb501a08f0 con 0x7fbb50073380 2023-08-25T04:44:38.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.587+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440053c0 con 0x7fbb50073380 2023-08-25T04:44:38.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.587+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb501a0cb0 con 0x7fbb50073380 2023-08-25T04:44:38.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.588+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50106e90 con 0x7fbb50073380 2023-08-25T04:44:38.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.590+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fbb44005580 con 0x7fbb50073380 2023-08-25T04:44:38.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.591+0000 7fbb4dffb700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 0x7fbb400a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:38.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.591+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbb440cc680 con 0x7fbb50073380 2023-08-25T04:44:38.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.591+0000 7fbb4ffff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 0x7fbb400a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:38.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.592+0000 7fbb4ffff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 0x7fbb400a2460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb50074680 tx=0x7fbb3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:38.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.595+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbb4405ae00 con 0x7fbb50073380 2023-08-25T04:44:38.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.799+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb5010e580 con 0x7fbb50073380 2023-08-25T04:44:38.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.801+0000 7fbb4dffb700 1 -- 172.21.15.145:0/1186300910 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7fbb440961a0 con 0x7fbb50073380 2023-08-25T04:44:38.801 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:38.802 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:38.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 msgr2=0x7fbb400a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 0x7fbb400a2460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb50074680 tx=0x7fbb3c009260 comp rx=0 tx=0).stop 2023-08-25T04:44:38.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 msgr2=0x7fbb50198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb44004740 tx=0x7fbb44009580 comp rx=0 tx=0).stop 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 shutdown_connections 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbb4009ffa0 0x7fbb400a2460 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.803+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb501087e0 0x7fbb5019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.804+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb50073380 0x7fbb50198820 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.804+0000 7fbb56fff700 1 --2- 172.21.15.145:0/1186300910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb50074960 0x7fbb50198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.804+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 >> 172.21.15.145:0/1186300910 conn(0x7fbb500fbe50 msgr2=0x7fbb500fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.804+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 shutdown_connections 2023-08-25T04:44:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:38.804+0000 7fbb56fff700 1 -- 172.21.15.145:0/1186300910 wait complete. 2023-08-25T04:44:38.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-25T04:42:49.862390+0000', 'modified': '2023-08-25T04:43:16.556417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24562, 'mds_1': 24557, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/557521219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 557521219}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 557521219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/4241662550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 4241662550}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 4241662550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6837/1048214793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1048214793}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1048214793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.183:6839/3401840698', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3401840698}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3401840698}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.162:6839/2860621511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 2860621511}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 2860621511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.145:6841/3986940142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6840', 'nonce': 3986940142}, {'type': 'v1', 'addr': '172.21.15.145:6841', 'nonce': 3986940142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T04:44:38.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-08-25T04:44:38.825 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-25T04:44:39.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 -- 172.21.15.145:0/1331821791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874100930 msgr2=0x7f5874100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1331821791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874100930 0x7f5874100d50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58640099c0 tx=0x7f5864009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 -- 172.21.15.145:0/1331821791 shutdown_connections 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1331821791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f5874108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1331821791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874100930 0x7f5874100d50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1331821791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874101b20 0x7f5874101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.021+0000 7f587a9ff700 1 -- 172.21.15.145:0/1331821791 >> 172.21.15.145:0/1331821791 conn(0x7f58740fbea0 msgr2=0x7f58740fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.022+0000 7f587a9ff700 1 -- 172.21.15.145:0/1331821791 shutdown_connections 2023-08-25T04:44:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.022+0000 7f587a9ff700 1 -- 172.21.15.145:0/1331821791 wait complete. 2023-08-25T04:44:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.022+0000 7f587a9ff700 1 Processor -- start 2023-08-25T04:44:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 -- start start 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 0x7f5874198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874101b20 0x7f5874198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58741a03d0 con 0x7f58741024c0 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58741a0510 con 0x7f5874101b20 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f587a9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58741a06a0 con 0x7f5874100930 2023-08-25T04:44:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f5878f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f5878f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50032/0 (socket says 172.21.15.145:50032) 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f5878f9c700 1 -- 172.21.15.145:0/1587668781 learned_addr learned my addr 172.21.15.145:0/1587668781 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f5873fff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 0x7f5874198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.023+0000 7f58737fe700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874101b20 0x7f5874198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 -- 172.21.15.145:0/1587668781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 msgr2=0x7f5874198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 0x7f5874198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 -- 172.21.15.145:0/1587668781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874101b20 msgr2=0x7f5874198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874101b20 0x7f5874198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 -- 172.21.15.145:0/1587668781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640095f0 con 0x7f58741024c0 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5873fff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 0x7f5874198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f5878f9c700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ef30 tx=0x7f586c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f58717fa700 1 -- 172.21.15.145:0/1587668781 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c018070 con 0x7f58741024c0 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f58717fa700 1 -- 172.21.15.145:0/1587668781 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586c00f040 con 0x7f58741024c0 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f58717fa700 1 -- 172.21.15.145:0/1587668781 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c00cd00 con 0x7f58741024c0 2023-08-25T04:44:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58741a0920 con 0x7f58741024c0 2023-08-25T04:44:39.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.024+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58741a0de0 con 0x7f58741024c0 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.027+0000 7f58717fa700 1 -- 172.21.15.145:0/1587668781 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f586c003a00 con 0x7f58741024c0 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.028+0000 7f588e41ab80 1 Processor -- start 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f58717fa700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 0x7f585c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 -- start start 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f58717fa700 1 -- 172.21.15.145:0/1587668781 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f586c0cb0f0 con 0x7f58741024c0 2023-08-25T04:44:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f58737fe700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 0x7f585c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f1432e0 0x55c36f143720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 0x55c36f144180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f0aa760 con 0x55c36f142980 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f09b730 con 0x55c36f1432e0 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f588e41ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f099050 con 0x55c36f143d00 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f5873fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f5873fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50036/0 (socket says 172.21.15.145:50036) 2023-08-25T04:44:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f5873fff700 1 -- 172.21.15.145:0/3942160744 learned_addr learned my addr 172.21.15.145:0/3942160744 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f58737fe700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f1432e0 0x55c36f143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.029+0000 7f5878f9c700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 0x55c36f144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f58737fe700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 0x7f585c0a2410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5868005950 tx=0x7f58680106b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 -- 172.21.15.145:0/3942160744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 msgr2=0x55c36f144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 0x55c36f144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 -- 172.21.15.145:0/3942160744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f1432e0 msgr2=0x55c36f143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f1432e0 0x55c36f143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 -- 172.21.15.145:0/3942160744 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c36ef145e0 con 0x55c36f142980 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5878f9c700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 0x55c36f144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f5873fff700 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58640099c0 tx=0x7f5864005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9500b1bfed2b05e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f585a7fc700 1 -- 172.21.15.145:0/3942160744 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586400bdc0 con 0x55c36f142980 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f585a7fc700 1 -- 172.21.15.145:0/3942160744 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586401e040 con 0x55c36f142980 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.030+0000 7f585a7fc700 1 -- 172.21.15.145:0/3942160744 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586400f460 con 0x55c36f142980 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 -- 172.21.15.145:0/3942160744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 msgr2=0x55c36f142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58640099c0 tx=0x7f5864005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 -- 172.21.15.145:0/3942160744 shutdown_connections 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f143d00 0x55c36f144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f142980 0x55c36f142da0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 --2- 172.21.15.145:0/3942160744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f1432e0 0x55c36f143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 -- 172.21.15.145:0/3942160744 >> 172.21.15.145:0/3942160744 conn(0x55c36f0b7090 msgr2=0x55c36f13d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 -- 172.21.15.145:0/3942160744 shutdown_connections 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.031+0000 7f588e41ab80 1 -- 172.21.15.145:0/3942160744 wait complete. 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 -- start start 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 Processor -- start 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f153840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f094940 con 0x55c36f1432e0 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f032b60 con 0x55c36f143d00 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.032+0000 7f588e41ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f032860 con 0x55c36f142980 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f588e41ab80 10 client.0 _ll_register_callbacks cb 0x55c36ef7ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f5873fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50048/0 (socket says 172.21.15.145:50048) 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 -- 172.21.15.145:0/198235530 learned_addr learned my addr 172.21.15.145:0/198235530 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f5878f9c700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 -- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 msgr2=0x55c36f14c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f14c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 -- 172.21.15.145:0/198235530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 msgr2=0x55c36f153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 -- 172.21.15.145:0/198235530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58741a0de0 con 0x55c36f1432e0 2023-08-25T04:44:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f5878f9c700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58737fe700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f586800be30 tx=0x7f58680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330b33ff7cde5d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.033+0000 7f58537fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868028640 con 0x55c36f1432e0 2023-08-25T04:44:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c36f112170 con 0x55c36f1432e0 2023-08-25T04:44:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5868019040 con 0x55c36f1432e0 2023-08-25T04:44:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58680132e0 con 0x55c36f1432e0 2023-08-25T04:44:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 10 client.15264 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f58741096c0 con 0x55c36f1432e0 2023-08-25T04:44:39.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c36f112170 con 0x55c36f1432e0 2023-08-25T04:44:39.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5868011790 con 0x55c36f1432e0 2023-08-25T04:44:39.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 10 client.15264 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T04:44:39.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 20 client.15264 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f157f10 0x55c36f1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 4 client.15264 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.034+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c36f03ab50 con 0x55c36f157f10 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.035+0000 7f5873fff700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f157f10 0x55c36f1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.035+0000 7f5873fff700 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f157f10 0x55c36f1569a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed207d4e2b86e7ea server_cookie=720d27eab33c4684 in_seq=0 out_seq=0 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.035+0000 7f58537fe700 10 client.15264 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.036+0000 7f58537fe700 1 -- 172.21.15.145:0/198235530 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c36f03ab50 con 0x55c36f157f10 2023-08-25T04:44:39.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.036+0000 7f58537fe700 10 client.15264 handle_command_reply: tid=0 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.036+0000 7f588e41ab80 1 client.15264 shutdown 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.036+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 msgr2=0x55c36f14c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.036+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f586800be30 tx=0x7f58680047c0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 shutdown_connections 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f157f10 0x55c36f1569a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f58537fe700 0 client.15264 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f14c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 --2- 172.21.15.145:0/198235530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f14c800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 >> 172.21.15.145:0/198235530 conn(0x55c36f0b7090 msgr2=0x55c36f137af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 shutdown_connections 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 1 -- 172.21.15.145:0/198235530 wait complete. 2023-08-25T04:44:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.037+0000 7f588e41ab80 20 client.15264 trim_cache size 0 max 16384 2023-08-25T04:44:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 Processor -- start 2023-08-25T04:44:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 -- start start 2023-08-25T04:44:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f142980 0x55c36f13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36ef168d0 con 0x55c36f143d00 2023-08-25T04:44:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f14b790 con 0x55c36f142980 2023-08-25T04:44:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f588e41ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36ef22de0 con 0x55c36f1432e0 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50050/0 (socket says 172.21.15.145:50050) 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 -- 172.21.15.145:0/2158097665 learned_addr learned my addr 172.21.15.145:0/2158097665 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5873fff700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f142980 0x55c36f13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f58737fe700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 -- 172.21.15.145:0/2158097665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 msgr2=0x55c36f153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 -- 172.21.15.145:0/2158097665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f142980 msgr2=0x55c36f13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f142980 0x55c36f13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f5878f9c700 1 -- 172.21.15.145:0/2158097665 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c36eec0d90 con 0x55c36f143d00 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.055+0000 7f58737fe700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f5878f9c700 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f586c0c8890 tx=0x7f586c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9051465b4e820be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f585affd700 1 -- 172.21.15.145:0/2158097665 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0abea0 con 0x55c36f143d00 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f585affd700 1 -- 172.21.15.145:0/2158097665 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586c01b320 con 0x55c36f143d00 2023-08-25T04:44:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f585affd700 1 -- 172.21.15.145:0/2158097665 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0a0680 con 0x55c36f143d00 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f588e41ab80 1 -- 172.21.15.145:0/2158097665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 msgr2=0x55c36f146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.056+0000 7f588e41ab80 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f586c0c8890 tx=0x7f586c0092e0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 -- 172.21.15.145:0/2158097665 shutdown_connections 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f143d00 0x55c36f146bc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f142980 0x55c36f13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 --2- 172.21.15.145:0/2158097665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 -- 172.21.15.145:0/2158097665 >> 172.21.15.145:0/2158097665 conn(0x55c36f0b7090 msgr2=0x55c36f136ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 -- 172.21.15.145:0/2158097665 shutdown_connections 2023-08-25T04:44:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 -- 172.21.15.145:0/2158097665 wait complete. 2023-08-25T04:44:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 -- start start 2023-08-25T04:44:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.057+0000 7f588e41ab80 1 Processor -- start 2023-08-25T04:44:39.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f137260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f1377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f149910 con 0x55c36f1432e0 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f137ce0 con 0x55c36f143d00 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c36f137ea0 con 0x55c36f142980 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f588e41ab80 10 client.0 _ll_register_callbacks cb 0x7f5868010db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50064/0 (socket says 172.21.15.145:50064) 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 -- 172.21.15.145:0/1958933244 learned_addr learned my addr 172.21.15.145:0/1958933244 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f5878f9c700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f1377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f5873fff700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f137260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 -- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 msgr2=0x55c36f137260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f137260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 -- 172.21.15.145:0/1958933244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 msgr2=0x55c36f1377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f1377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 -- 172.21.15.145:0/1958933244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58741a0920 con 0x55c36f1432e0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58737fe700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5868010ab0 tx=0x7f5868015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b34ca1400ac5bfe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.058+0000 7f58537fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868011940 con 0x55c36f1432e0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c36f135860 con 0x55c36f1432e0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5868019070 con 0x55c36f1432e0 2023-08-25T04:44:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58680132e0 con 0x55c36f1432e0 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 10 client.15270 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c36eea9c10 con 0x55c36f1432e0 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c36f135860 con 0x55c36f1432e0 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f586803d070 con 0x55c36f1432e0 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 10 client.15270 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 20 client.15270 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f1b0730 0x55c36f1acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 4 client.15270 mds_command: new command op to 24532 tid=0[{"prefix": "status"}] 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.059+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c36f08d920 con 0x55c36f1b0730 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f5873fff700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f1b0730 0x55c36f1acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f58537fe700 10 client.15270 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f5873fff700 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f1b0730 0x55c36f1acbb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71fbd276cf7f7890 server_cookie=44929aa757857635 in_seq=0 out_seq=0 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f58537fe700 1 -- 172.21.15.145:0/1958933244 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c36f08d920 con 0x55c36f1b0730 2023-08-25T04:44:39.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f58537fe700 10 client.15270 handle_command_reply: tid=0 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.060+0000 7f588e41ab80 1 client.15270 shutdown 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 msgr2=0x55c36f153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5868010ab0 tx=0x7f5868015000 comp rx=0 tx=0).stop 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 shutdown_connections 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55c36f1b0730 0x55c36f1acbb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c36f143d00 0x55c36f1377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c36f142980 0x55c36f137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 --2- 172.21.15.145:0/1958933244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c36f1432e0 0x55c36f153800 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f58537fe700 0 client.15270 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 >> 172.21.15.145:0/1958933244 conn(0x55c36f0b7090 msgr2=0x55c36f13d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 shutdown_connections 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 1 -- 172.21.15.145:0/1958933244 wait complete. 2023-08-25T04:44:39.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f588e41ab80 20 client.15270 trim_cache size 0 max 16384 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 2, 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24532, 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:39.071 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 83.506956110000004, 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 124.536343277 2023-08-25T04:44:39.072 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 msgr2=0x7f585c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.061+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 0x7f585c0a2410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5868005950 tx=0x7f58680106b0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 msgr2=0x7f587419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f586c00ef30 tx=0x7f586c00c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 shutdown_connections 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f585c09ff50 0x7f585c0a2410 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58741024c0 0x7f587419fe00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5874100930 0x7f5874198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 --2- 172.21.15.145:0/1587668781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5874101b20 0x7f5874198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 >> 172.21.15.145:0/1587668781 conn(0x7f58740fbea0 msgr2=0x7f58740fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 shutdown_connections 2023-08-25T04:44:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.062+0000 7f587a9ff700 1 -- 172.21.15.145:0/1587668781 wait complete. 2023-08-25T04:44:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:38 smithi145 ceph-mon[97681]: pgmap v321: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:38 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/212892547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:39.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:38 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3204741978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:39.080 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T04:44:39.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:38 smithi162 ceph-mon[130941]: pgmap v321: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:38 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/212892547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:39.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:38 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3204741978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:38 smithi183 ceph-mon[131414]: pgmap v321: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:44:39.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:38 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/212892547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:39.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:38 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3204741978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:39.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 -- 172.21.15.145:0/1005233807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc01087f0 msgr2=0x7facc010ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 --2- 172.21.15.145:0/1005233807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc01087f0 0x7facc010ac80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7facb800b0a0 tx=0x7facb800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 -- 172.21.15.145:0/1005233807 shutdown_connections 2023-08-25T04:44:39.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 --2- 172.21.15.145:0/1005233807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc01087f0 0x7facc010ac80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 --2- 172.21.15.145:0/1005233807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0074960 0x7facc0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 --2- 172.21.15.145:0/1005233807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc0073380 0x7facc0073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 -- 172.21.15.145:0/1005233807 >> 172.21.15.145:0/1005233807 conn(0x7facc00fbea0 msgr2=0x7facc00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 -- 172.21.15.145:0/1005233807 shutdown_connections 2023-08-25T04:44:39.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.278+0000 7facc69ff700 1 -- 172.21.15.145:0/1005233807 wait complete. 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 Processor -- start 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 -- start start 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 0x7facc01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc01087f0 0x7facc019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a0440 con 0x7facc0074960 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a0580 con 0x7facc01087f0 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.279+0000 7facc69ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a0710 con 0x7facc0073380 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50086/0 (socket says 172.21.15.145:50086) 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 -- 172.21.15.145:0/2808728875 learned_addr learned my addr 172.21.15.145:0/2808728875 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbffff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 0x7facc01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facc4f9c700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc01087f0 0x7facc019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 -- 172.21.15.145:0/2808728875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 msgr2=0x7facc01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 0x7facc01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 -- 172.21.15.145:0/2808728875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc01087f0 msgr2=0x7facc019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc01087f0 0x7facc019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 -- 172.21.15.145:0/2808728875 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009d00 con 0x7facc0074960 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbffff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 0x7facc01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.280+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7facb400e970 tx=0x7facb400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.281+0000 7facbd7fa700 1 -- 172.21.15.145:0/2808728875 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4009790 con 0x7facc0074960 2023-08-25T04:44:39.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.281+0000 7facbd7fa700 1 -- 172.21.15.145:0/2808728875 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb400c7e0 con 0x7facc0074960 2023-08-25T04:44:39.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.281+0000 7facbd7fa700 1 -- 172.21.15.145:0/2808728875 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4003c70 con 0x7facc0074960 2023-08-25T04:44:39.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.281+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc01a09f0 con 0x7facc0074960 2023-08-25T04:44:39.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.281+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01a0f00 con 0x7facc0074960 2023-08-25T04:44:39.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.284+0000 7facbd7fa700 1 -- 172.21.15.145:0/2808728875 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7facb400cc50 con 0x7facc0074960 2023-08-25T04:44:39.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.285+0000 7facda2bbb80 1 Processor -- start 2023-08-25T04:44:39.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.285+0000 7facbd7fa700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 0x7faca80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 -- start start 2023-08-25T04:44:39.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facbd7fa700 1 -- 172.21.15.145:0/2808728875 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7facb4014070 con 0x7facc0074960 2023-08-25T04:44:39.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facc4f9c700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 0x7faca80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dbff980 0x55829dbffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 0x55829dc01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829db67760 con 0x55829dc002e0 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829db58730 con 0x55829dbff980 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facda2bbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829db56050 con 0x55829dc00d00 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50092/0 (socket says 172.21.15.145:50092) 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facbf7fe700 1 -- 172.21.15.145:0/3028623275 learned_addr learned my addr 172.21.15.145:0/3028623275 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facc4f9c700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 0x55829dc01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.286+0000 7facbffff700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dbff980 0x55829dbffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 0x7faca80a2490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fd0 tx=0x7facb80094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 -- 172.21.15.145:0/3028623275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dbff980 msgr2=0x55829dbffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dbff980 0x55829dbffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 -- 172.21.15.145:0/3028623275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 msgr2=0x55829dc00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 -- 172.21.15.145:0/3028623275 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829d9d15e0 con 0x55829dc00d00 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facbf7fe700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7facc4f9c700 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 0x55829dc01180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facb8007a60 tx=0x7facb8024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c8b97acaf101a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7faca67fc700 1 -- 172.21.15.145:0/3028623275 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800e040 con 0x55829dc00d00 2023-08-25T04:44:39.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7faca67fc700 1 -- 172.21.15.145:0/3028623275 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb8003e60 con 0x55829dc00d00 2023-08-25T04:44:39.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.287+0000 7faca67fc700 1 -- 172.21.15.145:0/3028623275 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8029680 con 0x55829dc00d00 2023-08-25T04:44:39.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 -- 172.21.15.145:0/3028623275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 msgr2=0x55829dc01180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 0x55829dc01180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facb8007a60 tx=0x7facb8024000 comp rx=0 tx=0).stop 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 -- 172.21.15.145:0/3028623275 shutdown_connections 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc00d00 0x55829dc01180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dbff980 0x55829dbffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3028623275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc002e0 0x55829dc00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 -- 172.21.15.145:0/3028623275 >> 172.21.15.145:0/3028623275 conn(0x55829db74090 msgr2=0x55829dbfa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 -- 172.21.15.145:0/3028623275 shutdown_connections 2023-08-25T04:44:39.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.288+0000 7facda2bbb80 1 -- 172.21.15.145:0/3028623275 wait complete. 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.289+0000 7facda2bbb80 1 -- start start 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.289+0000 7facda2bbb80 1 Processor -- start 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829db51940 con 0x55829dc00d00 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829daefb60 con 0x55829dc002e0 2023-08-25T04:44:39.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829daef860 con 0x55829dbff980 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facda2bbb80 10 client.0 _ll_register_callbacks cb 0x55829da39a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facc4f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41832/0 (socket says 172.21.15.145:41832) 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 -- 172.21.15.145:0/2152940574 learned_addr learned my addr 172.21.15.145:0/2152940574 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 -- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 msgr2=0x55829dc09800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc09800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 -- 172.21.15.145:0/2152940574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 msgr2=0x55829dc10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facbffff700 1 -- 172.21.15.145:0/2152940574 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01a0f00 con 0x55829dbff980 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.290+0000 7facc4f9c700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7facbffff700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facb0012d20 tx=0x7facb000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321779cbfe1d4fa6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:44:39.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0017070 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55829dbcf170 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7facb000fb80 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb00122e0 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7facda2bbb80 10 client.25291 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7facc010b8a0 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55829dbcf170 con 0x55829dbff980 2023-08-25T04:44:39.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.291+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7facb001c420 con 0x55829dbff980 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 10 client.25291 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 10 client.25291 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 20 client.25291 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc14f20 0x55829dc139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 4 client.25291 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55829daf7b50 con 0x55829dc14f20 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc14f20 0x55829dc139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7fac9f7fe700 10 client.25291 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:39.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.292+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc14f20 0x55829dc139a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ddc0e26fcdacf06 server_cookie=fcdd0526318c136 in_seq=0 out_seq=0 2023-08-25T04:44:39.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.293+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2152940574 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55829daf7b50 con 0x55829dc14f20 2023-08-25T04:44:39.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.293+0000 7fac9f7fe700 10 client.25291 handle_command_reply: tid=0 2023-08-25T04:44:39.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.293+0000 7facda2bbb80 1 client.25291 shutdown 2023-08-25T04:44:39.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 msgr2=0x55829dc092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facb0012d20 tx=0x7facb000f6a0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 shutdown_connections 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc14f20 0x55829dc139a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7fac9f7fe700 0 client.25291 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc092c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2152940574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 >> 172.21.15.145:0/2152940574 conn(0x55829db74090 msgr2=0x55829dbf4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 shutdown_connections 2023-08-25T04:44:39.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 1 -- 172.21.15.145:0/2152940574 wait complete. 2023-08-25T04:44:39.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.294+0000 7facda2bbb80 20 client.25291 trim_cache size 0 max 16384 2023-08-25T04:44:39.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 Processor -- start 2023-08-25T04:44:39.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 -- start start 2023-08-25T04:44:39.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 0x55829dc10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc00d00 0x55829dc03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829d9d38d0 con 0x55829dbff980 2023-08-25T04:44:39.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829dc078e0 con 0x55829dc00d00 2023-08-25T04:44:39.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facda2bbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829d9dfde0 con 0x55829dc002e0 2023-08-25T04:44:39.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facbffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facbffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50114/0 (socket says 172.21.15.145:50114) 2023-08-25T04:44:39.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.312+0000 7facbffff700 1 -- 172.21.15.145:0/3329454029 learned_addr learned my addr 172.21.15.145:0/3329454029 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbf7fe700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 0x55829dc10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facc4f9c700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc00d00 0x55829dc03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 -- 172.21.15.145:0/3329454029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 msgr2=0x55829dc10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 0x55829dc10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 -- 172.21.15.145:0/3329454029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc00d00 msgr2=0x55829dc03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc00d00 0x55829dc03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 -- 172.21.15.145:0/3329454029 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829d97dd90 con 0x55829dbff980 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbf7fe700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 0x55829dc10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7facbffff700 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d58b81ae1f54f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7faca6ffd700 1 -- 172.21.15.145:0/3329454029 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0017030 con 0x55829dbff980 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7faca6ffd700 1 -- 172.21.15.145:0/3329454029 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb0018040 con 0x55829dbff980 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.313+0000 7faca6ffd700 1 -- 172.21.15.145:0/3329454029 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0005510 con 0x55829dbff980 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- 172.21.15.145:0/3329454029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 msgr2=0x55829dc083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb0005e40 comp rx=0 tx=0).stop 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- 172.21.15.145:0/3329454029 shutdown_connections 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc00d00 0x55829dc03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dbff980 0x55829dc083c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 --2- 172.21.15.145:0/3329454029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dc002e0 0x55829dc10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- 172.21.15.145:0/3329454029 >> 172.21.15.145:0/3329454029 conn(0x55829db74090 msgr2=0x55829dbfa200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- 172.21.15.145:0/3329454029 shutdown_connections 2023-08-25T04:44:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- 172.21.15.145:0/3329454029 wait complete. 2023-08-25T04:44:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.314+0000 7facda2bbb80 1 -- start start 2023-08-25T04:44:39.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 Processor -- start 2023-08-25T04:44:39.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc0d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829dc107a0 con 0x55829dc00d00 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829dc10910 con 0x55829dc002e0 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829dc10ad0 con 0x55829dbff980 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facda2bbb80 10 client.0 _ll_register_callbacks cb 0x7facb00127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facbffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc0d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47834/0 (socket says 172.21.15.145:47834) 2023-08-25T04:44:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facbf7fe700 1 -- 172.21.15.145:0/2425878682 learned_addr learned my addr 172.21.15.145:0/2425878682 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.315+0000 7facc4f9c700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 -- 172.21.15.145:0/2425878682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 msgr2=0x55829dc0d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc0d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 -- 172.21.15.145:0/2425878682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 msgr2=0x55829dc10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 -- 172.21.15.145:0/2425878682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc010b8a0 con 0x55829dc002e0 2023-08-25T04:44:39.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbffff700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc0d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facbf7fe700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facb400efd0 tx=0x7facb4003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aedabb327907b4a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7fac9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb409cbe0 con 0x55829dc002e0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55829dbf2860 con 0x55829dc002e0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7facb40a0bc0 con 0x55829dc002e0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb40cb3a0 con 0x55829dc002e0 2023-08-25T04:44:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facda2bbb80 10 client.25289 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.316+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55829d966c10 con 0x55829dc002e0 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55829dbf2860 con 0x55829dc002e0 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7facb40b80b0 con 0x55829dc002e0 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 10 client.25289 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 10 client.25289 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 20 client.25289 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc6d840 0x55829dc69ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 4 client.25289 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-08-25T04:44:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55829db4a920 con 0x55829dc6d840 2023-08-25T04:44:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.317+0000 7facbffff700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc6d840 0x55829dc69ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facbffff700 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc6d840 0x55829dc69ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66233ba08db40ee8 server_cookie=44b3642e281fcc5d in_seq=0 out_seq=0 2023-08-25T04:44:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7fac9f7fe700 10 client.25289 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7fac9f7fe700 1 -- 172.21.15.145:0/2425878682 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55829db4a920 con 0x55829dc6d840 2023-08-25T04:44:39.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7fac9f7fe700 10 client.25289 handle_command_reply: tid=0 2023-08-25T04:44:39.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 client.25289 shutdown 2023-08-25T04:44:39.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 msgr2=0x55829dc0fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facb400efd0 tx=0x7facb4003710 comp rx=0 tx=0).stop 2023-08-25T04:44:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 shutdown_connections 2023-08-25T04:44:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55829dc6d840 0x55829dc69ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55829dc00d00 0x55829dc10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7fac9f7fe700 0 client.25289 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:44:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55829dbff980 0x55829dc0d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 --2- 172.21.15.145:0/2425878682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55829dc002e0 0x55829dc0fce0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 >> 172.21.15.145:0/2425878682 conn(0x55829db74090 msgr2=0x55829dc03b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 shutdown_connections 2023-08-25T04:44:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 1 -- 172.21.15.145:0/2425878682 wait complete. 2023-08-25T04:44:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.318+0000 7facda2bbb80 20 client.25289 trim_cache size 0 max 16384 2023-08-25T04:44:39.333 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 1, 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24557, 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:39.334 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 84.776973329, 2023-08-25T04:44:39.335 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:39.335 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:39.335 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:39.335 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 116.473569669 2023-08-25T04:44:39.335 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 msgr2=0x7faca80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 0x7faca80a2490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fd0 tx=0x7facb80094a0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 msgr2=0x7facc0198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7facb400e970 tx=0x7facb400ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 shutdown_connections 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faca809ffd0 0x7faca80a2490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facc01087f0 0x7facc019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc0073380 0x7facc01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 --2- 172.21.15.145:0/2808728875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facc0074960 0x7facc0198e00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 >> 172.21.15.145:0/2808728875 conn(0x7facc00fbea0 msgr2=0x7facc00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 shutdown_connections 2023-08-25T04:44:39.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.319+0000 7facc69ff700 1 -- 172.21.15.145:0/2808728875 wait complete. 2023-08-25T04:44:39.339 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T04:44:39.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 -- 172.21.15.145:0/3082885981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 msgr2=0x7f6194100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 --2- 172.21.15.145:0/3082885981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194100d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f61900099c0 tx=0x7f6190009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 -- 172.21.15.145:0/3082885981 shutdown_connections 2023-08-25T04:44:39.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 --2- 172.21.15.145:0/3082885981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 0x7f6194108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 --2- 172.21.15.145:0/3082885981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194100d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 --2- 172.21.15.145:0/3082885981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 -- 172.21.15.145:0/3082885981 >> 172.21.15.145:0/3082885981 conn(0x7f61940fbe60 msgr2=0x7f61940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 -- 172.21.15.145:0/3082885981 shutdown_connections 2023-08-25T04:44:39.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.535+0000 7f619c5ff700 1 -- 172.21.15.145:0/3082885981 wait complete. 2023-08-25T04:44:39.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.536+0000 7f619c5ff700 1 Processor -- start 2023-08-25T04:44:39.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.536+0000 7f619c5ff700 1 -- start start 2023-08-25T04:44:39.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 0x7f619419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419e1d0 con 0x7f6194100910 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419e310 con 0x7f61941024a0 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619c5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419e4a0 con 0x7f6194101b00 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50138/0 (socket says 172.21.15.145:50138) 2023-08-25T04:44:39.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 -- 172.21.15.145:0/1781980669 learned_addr learned my addr 172.21.15.145:0/1781980669 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f6199b9a700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619ab9c700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 0x7f619419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 -- 172.21.15.145:0/1781980669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 msgr2=0x7f6194196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 -- 172.21.15.145:0/1781980669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 msgr2=0x7f619419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 0x7f619419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f619a39b700 1 -- 172.21.15.145:0/1781980669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009520 con 0x7f6194100910 2023-08-25T04:44:39.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.537+0000 7f6199b9a700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f619a39b700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6190009fd0 tx=0x7f619000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f618b7fe700 1 -- 172.21.15.145:0/1781980669 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619001d070 con 0x7f6194100910 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f618b7fe700 1 -- 172.21.15.145:0/1781980669 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6190005510 con 0x7f6194100910 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61900095f0 con 0x7f6194100910 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f618b7fe700 1 -- 172.21.15.145:0/1781980669 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190004bb0 con 0x7f6194100910 2023-08-25T04:44:39.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.538+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619419e9b0 con 0x7f6194100910 2023-08-25T04:44:39.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.541+0000 7f618b7fe700 1 -- 172.21.15.145:0/1781980669 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f6190004d50 con 0x7f6194100910 2023-08-25T04:44:39.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.542+0000 7f618b7fe700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 0x7f61800a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.542+0000 7f61afe3ab80 1 Processor -- start 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.542+0000 7f618b7fe700 1 -- 172.21.15.145:0/1781980669 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f61900cc410 con 0x7f6194100910 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.542+0000 7f61afe3ab80 1 -- start start 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f6199b9a700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 0x7f61800a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 0x55a60dedfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dee0110 0x55a60dee0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60de42760 con 0x55a60deded90 2023-08-25T04:44:39.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60de33730 con 0x55a60dedf6f0 2023-08-25T04:44:39.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60de31050 con 0x55a60dee0110 2023-08-25T04:44:39.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50154/0 (socket says 172.21.15.145:50154) 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 -- 172.21.15.145:0/2846143914 learned_addr learned my addr 172.21.15.145:0/2846143914 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619ab9c700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dee0110 0x55a60dee0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f6199b9a700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 0x55a60dedfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f6199b9a700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 0x7f61800a2410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e90 tx=0x7f6184009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 -- 172.21.15.145:0/2846143914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dee0110 msgr2=0x55a60dee0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dee0110 0x55a60dee0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 -- 172.21.15.145:0/2846143914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 msgr2=0x55a60dedfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 0x55a60dedfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f619a39b700 1 -- 172.21.15.145:0/2846143914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a60dcac5e0 con 0x55a60deded90 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.543+0000 7f6199b9a700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 0x55a60dedfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.544+0000 7f619a39b700 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61900a13e0 tx=0x7f6190099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf82ea07f9d611a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.544+0000 7f6188bfe700 1 -- 172.21.15.145:0/2846143914 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190099b90 con 0x55a60deded90 2023-08-25T04:44:39.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.544+0000 7f6188bfe700 1 -- 172.21.15.145:0/2846143914 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619009d9c0 con 0x55a60deded90 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.544+0000 7f6188bfe700 1 -- 172.21.15.145:0/2846143914 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619009a050 con 0x55a60deded90 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2846143914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 msgr2=0x55a60dedf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61900a13e0 tx=0x7f6190099800 comp rx=0 tx=0).stop 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2846143914 shutdown_connections 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dee0110 0x55a60dee0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dedf1b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2846143914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dedf6f0 0x55a60dedfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2846143914 >> 172.21.15.145:0/2846143914 conn(0x55a60de4f090 msgr2=0x55a60ded5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2846143914 shutdown_connections 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2846143914 wait complete. 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 -- start start 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.545+0000 7f61afe3ab80 1 Processor -- start 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60dee8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.551 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60de2c940 con 0x55a60deded90 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60ddcab60 con 0x55a60dee0110 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60ddca860 con 0x55a60dedf6f0 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f61afe3ab80 10 client.0 _ll_register_callbacks cb 0x55a60dd14a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f619a39b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50168/0 (socket says 172.21.15.145:50168) 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f619a39b700 1 -- 172.21.15.145:0/587774999 learned_addr learned my addr 172.21.15.145:0/587774999 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f6199b9a700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60dee8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.546+0000 7f619ab9c700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 -- 172.21.15.145:0/587774999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 msgr2=0x55a60dee8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60dee8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 -- 172.21.15.145:0/587774999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 msgr2=0x55a60deef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 -- 172.21.15.145:0/587774999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619419e9b0 con 0x55a60deded90 2023-08-25T04:44:39.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f6199b9a700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60dee8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f619a39b700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f619000b580 tx=0x7f61900cc320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d64c88eb1c3968d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f617dffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900b1ea0 con 0x55a60deded90 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6190099b90 con 0x55a60deded90 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a60deaa170 con 0x55a60deded90 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6190099ff0 con 0x55a60deded90 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f61afe3ab80 10 client.15300 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.547+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a60dc58d90 con 0x55a60deded90 2023-08-25T04:44:39.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a60deaa170 con 0x55a60deded90 2023-08-25T04:44:39.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f619009aba0 con 0x55a60deded90 2023-08-25T04:44:39.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 10 client.15300 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 10 client.15300 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:44:39.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 20 client.15300 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60def3bc0 0x55a60def2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 4 client.15300 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a60ddd2b50 con 0x55a60def3bc0 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.548+0000 7f6199b9a700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60def3bc0 0x55a60def2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f617dffb700 10 client.15300 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f6199b9a700 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60def3bc0 0x55a60def2900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc7e2369a8bf543 server_cookie=9fe38948a70879a7 in_seq=0 out_seq=0 2023-08-25T04:44:39.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f617dffb700 1 -- 172.21.15.145:0/587774999 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a60ddd2b50 con 0x55a60def3bc0 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f617dffb700 10 client.15300 handle_command_reply: tid=0 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f61afe3ab80 1 client.15300 shutdown 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.549+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 msgr2=0x55a60dee8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f619000b580 tx=0x7f61900cc320 comp rx=0 tx=0).stop 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 shutdown_connections 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60deded90 0x55a60dee8450 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f619000b580 tx=0x7f61900cc320 comp rx=0 tx=0).stop 2023-08-25T04:44:39.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60def3bc0 0x55a60def2900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/587774999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60dee8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 >> 172.21.15.145:0/587774999 conn(0x55a60de4f090 msgr2=0x55a60decfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f617dffb700 0 client.15300 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 shutdown_connections 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 1 -- 172.21.15.145:0/587774999 wait complete. 2023-08-25T04:44:39.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.550+0000 7f61afe3ab80 20 client.15300 trim_cache size 0 max 16384 2023-08-25T04:44:39.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.568+0000 7f61afe3ab80 1 Processor -- start 2023-08-25T04:44:39.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.568+0000 7f61afe3ab80 1 -- start start 2023-08-25T04:44:39.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60decead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60def2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60dcae8d0 con 0x55a60dee64c0 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60ddd4470 con 0x55a60dee0110 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60dcbade0 con 0x55a60dedf6f0 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f6199b9a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f6199b9a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47882/0 (socket says 172.21.15.145:47882) 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f6199b9a700 1 -- 172.21.15.145:0/4265696434 learned_addr learned my addr 172.21.15.145:0/4265696434 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f619ab9c700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60def2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.569+0000 7f619a39b700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60decead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 -- 172.21.15.145:0/4265696434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 msgr2=0x55a60decead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60decead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 -- 172.21.15.145:0/4265696434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 msgr2=0x55a60def2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60def2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 -- 172.21.15.145:0/4265696434 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a60dc41c10 con 0x55a60dee0110 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f619ab9c700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60def2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f619a39b700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60decead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:39.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f6199b9a700 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6184009d50 tx=0x7f6184014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392caf48e49dc40c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f61893ff700 1 -- 172.21.15.145:0/4265696434 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184012ea0 con 0x55a60dee0110 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f61893ff700 1 -- 172.21.15.145:0/4265696434 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6184027040 con 0x55a60dee0110 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.570+0000 7f61893ff700 1 -- 172.21.15.145:0/4265696434 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184003680 con 0x55a60dee0110 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- 172.21.15.145:0/4265696434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 msgr2=0x55a60deef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6184009d50 tx=0x7f6184014000 comp rx=0 tx=0).stop 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- 172.21.15.145:0/4265696434 shutdown_connections 2023-08-25T04:44:39.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60def2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60decead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/4265696434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deef960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- 172.21.15.145:0/4265696434 >> 172.21.15.145:0/4265696434 conn(0x55a60de4f090 msgr2=0x55a60dee7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- 172.21.15.145:0/4265696434 shutdown_connections 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- 172.21.15.145:0/4265696434 wait complete. 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.571+0000 7f61afe3ab80 1 -- start start 2023-08-25T04:44:39.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 Processor -- start 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60deec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deeee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60deefa90 con 0x55a60dee64c0 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60deefc00 con 0x55a60dee0110 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60deefdc0 con 0x55a60dedf6f0 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f61afe3ab80 10 client.0 _ll_register_callbacks cb 0x7f618c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:39.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f619ab9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f619ab9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50190/0 (socket says 172.21.15.145:50190) 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f619ab9c700 1 -- 172.21.15.145:0/2334563296 learned_addr learned my addr 172.21.15.145:0/2334563296 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f619a39b700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60deec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.572+0000 7f6199b9a700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deeee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 -- 172.21.15.145:0/2334563296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 msgr2=0x55a60deec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60deec920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 -- 172.21.15.145:0/2334563296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 msgr2=0x55a60deeee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:39.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deeee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 -- 172.21.15.145:0/2334563296 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a60dc58d90 con 0x55a60dee64c0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f619ab9c700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f618c007f10 tx=0x7f618c0176f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321edd8648489317 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f617dffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00e030 con 0x55a60dee64c0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f618c003c00 con 0x55a60dee64c0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.573+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a60decd860 con 0x55a60dee64c0 2023-08-25T04:44:39.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f618c003dc0 con 0x55a60dee64c0 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 10 client.15306 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f618c009190 con 0x55a60dee64c0 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a60decd860 con 0x55a60dee64c0 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f618c004260 con 0x55a60dee64c0 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 10 client.15306 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 10 client.15306 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 20 client.15306 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:39.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60df4c6a0 0x55a60df48bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:39.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 4 client.15306 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-08-25T04:44:39.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a60de25920 con 0x55a60df4c6a0 2023-08-25T04:44:39.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.574+0000 7f619a39b700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60df4c6a0 0x55a60df48bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:39.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f619a39b700 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60df4c6a0 0x55a60df48bc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d611bd57f5082f82 server_cookie=4784e39c2ce3ef0d in_seq=0 out_seq=0 2023-08-25T04:44:39.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f617dffb700 10 client.15306 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:39.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f617dffb700 1 -- 172.21.15.145:0/2334563296 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55a60de25920 con 0x55a60df4c6a0 2023-08-25T04:44:39.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f617dffb700 10 client.15306 handle_command_reply: tid=0 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 client.15306 shutdown 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 msgr2=0x55a60deef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f618c007f10 tx=0x7f618c0176f0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 shutdown_connections 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a60dee64c0 0x55a60deef3b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a60df4c6a0 0x55a60df48bc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a60dedf6f0 0x55a60deec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 --2- 172.21.15.145:0/2334563296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a60dee0110 0x55a60deeee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f617dffb700 0 client.15306 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 >> 172.21.15.145:0/2334563296 conn(0x55a60de4f090 msgr2=0x55a60decead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 shutdown_connections 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.575+0000 7f61afe3ab80 1 -- 172.21.15.145:0/2334563296 wait complete. 2023-08-25T04:44:39.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f61afe3ab80 20 client.15306 trim_cache size 0 max 16384 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "b24d70d4-4300-11ee-9b3d-001a4aab830c", 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 24562, 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2023-08-25T04:44:39.585 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 108.756714819, 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 23, 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 89, 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 87, 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 119.002388025 2023-08-25T04:44:39.586 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:39.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 msgr2=0x7f61800a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 0x7f61800a2410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e90 tx=0x7f6184009260 comp rx=0 tx=0).stop 2023-08-25T04:44:39.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 msgr2=0x7f6194196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:39.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6190009fd0 tx=0x7f619000f740 comp rx=0 tx=0).stop 2023-08-25T04:44:39.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 shutdown_connections 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f618009ff50 0x7f61800a2410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61941024a0 0x7f619419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6194100910 0x7f6194196650 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 --2- 172.21.15.145:0/1781980669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6194101b00 0x7f6194196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 >> 172.21.15.145:0/1781980669 conn(0x7f61940fbe60 msgr2=0x7f61940fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 shutdown_connections 2023-08-25T04:44:39.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:39.576+0000 7f619c5ff700 1 -- 172.21.15.145:0/1781980669 wait complete. 2023-08-25T04:44:39.595 INFO:teuthology.run_tasks:Running task kclient... 2023-08-25T04:44:39.608 INFO:tasks.kclient:Mounting kernel clients... 2023-08-25T04:44:39.608 INFO:tasks.kclient:config is {'client.0': {}} 2023-08-25T04:44:39.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:39.609 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2023-08-25T04:44:39.652 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-25T04:44:39.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:39.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-25T04:44:39.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:39.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2023-08-25T04:44:39.864 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:39 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1186300910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:39.874 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mountpoint': '/tmp/cephfs'} 2023-08-25T04:44:39.874 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T04:44:39.875 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-08-25T04:44:39.875 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-08-25T04:44:39.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph fs ls 2023-08-25T04:44:40.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:39 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1186300910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:39 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1186300910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:40.601 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T04:44:41.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:40 smithi145 ceph-mon[97681]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:41.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:40 smithi162 ceph-mon[130941]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:41.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:40 smithi183 ceph-mon[131414]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:44:41.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 -- 172.21.15.145:0/3925804023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30100940 msgr2=0x7fbe30100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:41.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 --2- 172.21.15.145:0/3925804023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30100940 0x7fbe30100dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009990 tx=0x7fbe24009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 -- 172.21.15.145:0/3925804023 shutdown_connections 2023-08-25T04:44:41.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 --2- 172.21.15.145:0/3925804023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30101300 0x7fbe3010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 --2- 172.21.15.145:0/3925804023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 --2- 172.21.15.145:0/3925804023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30100940 0x7fbe30100dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 -- 172.21.15.145:0/3925804023 >> 172.21.15.145:0/3925804023 conn(0x7fbe300fc410 msgr2=0x7fbe300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:41.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 -- 172.21.15.145:0/3925804023 shutdown_connections 2023-08-25T04:44:41.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.969+0000 7fbe35499700 1 -- 172.21.15.145:0/3925804023 wait complete. 2023-08-25T04:44:41.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.970+0000 7fbe35499700 1 Processor -- start 2023-08-25T04:44:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.970+0000 7fbe35499700 1 -- start start 2023-08-25T04:44:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30199130 con 0x7fbe30101300 2023-08-25T04:44:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301992a0 con 0x7fbe30106940 2023-08-25T04:44:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe35499700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30199460 con 0x7fbe30100940 2023-08-25T04:44:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2e7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2e7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50208/0 (socket says 172.21.15.145:50208) 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41944/0 (socket says 172.21.15.145:41944) 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2e7fc700 1 -- 172.21.15.145:0/2718732237 learned_addr learned my addr 172.21.15.145:0/2718732237 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.971+0000 7fbe2f7fe700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 -- 172.21.15.145:0/2718732237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 msgr2=0x7fbe30198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 -- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 msgr2=0x7fbe3019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 -- 172.21.15.145:0/2718732237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240095f0 con 0x7fbe30100940 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2f7fe700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2e7fc700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe2effd700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e9f0 tx=0x7fbe2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:41.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000c9c0 con 0x7fbe30100940 2023-08-25T04:44:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe20010430 con 0x7fbe30100940 2023-08-25T04:44:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20018680 con 0x7fbe30100940 2023-08-25T04:44:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe30199740 con 0x7fbe30100940 2023-08-25T04:44:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.972+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe301a5a30 con 0x7fbe30100940 2023-08-25T04:44:41.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.973+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3004e620 con 0x7fbe30100940 2023-08-25T04:44:41.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.975+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fbe20010860 con 0x7fbe30100940 2023-08-25T04:44:41.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.976+0000 7fbe17fff700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 0x7fbe180a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:41.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.977+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbe20014070 con 0x7fbe30100940 2023-08-25T04:44:41.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.977+0000 7fbe2e7fc700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 0x7fbe180a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:41.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.977+0000 7fbe2e7fc700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 0x7fbe180a2410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe30106000 tx=0x7fbe2400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:41.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:41.981+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbe20051e00 con 0x7fbe30100940 2023-08-25T04:44:42.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.220+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe300008d0 con 0x7fbe30100940 2023-08-25T04:44:42.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.221+0000 7fbe17fff700 1 -- 172.21.15.145:0/2718732237 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbe20095020 con 0x7fbe30100940 2023-08-25T04:44:42.221 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-25T04:44:42.224 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 msgr2=0x7fbe180a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:42.224 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 0x7fbe180a2410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe30106000 tx=0x7fbe2400b260 comp rx=0 tx=0).stop 2023-08-25T04:44:42.224 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 msgr2=0x7fbe3019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e9f0 tx=0x7fbe2000ed00 comp rx=0 tx=0).stop 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 shutdown_connections 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fbe1809ff50 0x7fbe180a2410 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30106940 0x7fbe30198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe30100940 0x7fbe3019ea40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 --2- 172.21.15.145:0/2718732237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe30101300 0x7fbe3019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:42.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.223+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 >> 172.21.15.145:0/2718732237 conn(0x7fbe300fc410 msgr2=0x7fbe300fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:42.226 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.224+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 shutdown_connections 2023-08-25T04:44:42.226 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:42.224+0000 7fbe35499700 1 -- 172.21.15.145:0/2718732237 wait complete. 2023-08-25T04:44:42.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T04:44:42.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T04:44:42.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2023-08-25T04:44:42.788 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T04:44:42.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-08-25T04:44:42.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T04:44:42.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T04:44:42.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T04:44:42.790 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-08-25T04:44:42.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:42.791 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2023-08-25T04:44:42.836 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:44:42.837 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T04:44:42.838 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2023-08-25T04:44:42.839 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T04:44:42.839 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 41591sec preferred_lft 41591sec 2023-08-25T04:44:42.839 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::71eb:8a0d:636e:f0c3/64 scope link noprefixroute 2023-08-25T04:44:42.839 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2023-08-25T04:44:42.839 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T04:44:42.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:42.840 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:44:42.840 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:44:42.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add name ceph-brx type bridge 2023-08-25T04:44:42.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr flush dev ceph-brx 2023-08-25T04:44:42.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set ceph-brx up 2023-08-25T04:44:42.841 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T04:44:42.841 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:44:42.978 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:42 smithi145 ceph-mon[97681]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:42.979 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:42 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2718732237' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:44:43.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.087 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T04:44:43.161 INFO:teuthology.orchestra.run.smithi145.stdout:1 2023-08-25T04:44:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:42 smithi162 ceph-mon[130941]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:43.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:42 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2718732237' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:44:43.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.164 DEBUG:teuthology.orchestra.run.smithi145:> route 2023-08-25T04:44:43.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:42 smithi183 ceph-mon[131414]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:43.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:42 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2718732237' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-25T04:44:43.211 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2023-08-25T04:44:43.212 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T04:44:43.212 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T04:44:43.213 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T04:44:43.213 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T04:44:43.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.214 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:44:43.215 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:44:43.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T04:44:43.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T04:44:43.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T04:44:43.220 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:44:43.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.425 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2023-08-25T04:44:43.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.470 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2023-08-25T04:44:43.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.555 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:44:43.555 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:44:43.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--tmp-cephfs 2023-08-25T04:44:43.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-08-25T04:44:43.556 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:44:43.738 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-08-25T04:44:43.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.739 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:44:43.740 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:44:43.740 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-08-25T04:44:43.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T04:44:43.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-08-25T04:44:43.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-08-25T04:44:43.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-08-25T04:44:43.743 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:44:43.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:43.992 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec bash -c ' 2023-08-25T04:44:43.992 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2023-08-25T04:44:43.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2023-08-25T04:44:43.992 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T04:44:43.993 DEBUG:teuthology.orchestra.run.smithi145:> ') 2023-08-25T04:44:44.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T04:44:44.166 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /tmp/cephfs 2023-08-25T04:44:44.209 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/tmp/cephfs' 2023-08-25T04:44:44.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T04:44:44.210 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /tmp/cephfs 2023-08-25T04:44:44.292 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-08-25T04:44:44.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:44.293 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-08-25T04:44:44.438 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-08-25T04:44:44.438 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-08-25T04:44:44.901 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:44 smithi145 ceph-mon[97681]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-08-25T04:44:44.914 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-08-25T04:44:44.915 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-08-25T04:44:44.915 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.145:6789,172.21.15.162:6789,172.21.15.183:6789" 2023-08-25T04:44:44.915 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 0@b24d70d4-4300-11ee-9b3d-001a4aab830c.cephfs=/ 2023-08-25T04:44:44.915 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.145:6789/172.21.15.162:6789/172.21.15.183:6789" will pass to kernel 2023-08-25T04:44:44.920 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T04:44:44.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:44.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /tmp/cephfs 2023-08-25T04:44:44.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:44.991 DEBUG:teuthology.orchestra.run.smithi145:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T04:44:45.077 INFO:teuthology.orchestra.run.smithi145.stdout:client25324 2023-08-25T04:44:45.078 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T04:44:45.078 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T04:44:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:45.124 DEBUG:teuthology.orchestra.run.smithi145:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T04:44:45.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:44 smithi162 ceph-mon[130941]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-08-25T04:44:45.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:44 smithi183 ceph-mon[131414]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-08-25T04:44:45.248 INFO:teuthology.orchestra.run.smithi145.stdout:client25324 2023-08-25T04:44:45.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:45.249 DEBUG:teuthology.orchestra.run.smithi145:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T04:44:45.334 INFO:teuthology.orchestra.run.smithi145.stdout:client25324 2023-08-25T04:44:45.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T04:44:45.335 DEBUG:teuthology.orchestra.run.smithi145:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c.client25324/status') 2023-08-25T04:44:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:instance: client.25324 (3)172.21.15.145:0/2176333871 2023-08-25T04:44:45.442 INFO:teuthology.orchestra.run.smithi145.stdout:blocklisted: false 2023-08-25T04:44:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records in 2023-08-25T04:44:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records out 2023-08-25T04:44:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:72 bytes copied, 5.2084e-05 s, 1.4 MB/s 2023-08-25T04:44:45.445 INFO:teuthology.run_tasks:Running task check-counter... 2023-08-25T04:44:45.463 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-08-25T04:44:45.490 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:45.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 -- 172.21.15.145:0/3258864586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90100930 msgr2=0x7f4a90100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:45.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 --2- 172.21.15.145:0/3258864586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90100930 0x7f4a90100d50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0099c0 tx=0x7f4a8c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 -- 172.21.15.145:0/3258864586 shutdown_connections 2023-08-25T04:44:45.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 --2- 172.21.15.145:0/3258864586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 0x7f4a90108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 --2- 172.21.15.145:0/3258864586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90100930 0x7f4a90100d50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 --2- 172.21.15.145:0/3258864586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90101b20 0x7f4a90101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 -- 172.21.15.145:0/3258864586 >> 172.21.15.145:0/3258864586 conn(0x7f4a900fbe60 msgr2=0x7f4a900fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:45.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 -- 172.21.15.145:0/3258864586 shutdown_connections 2023-08-25T04:44:45.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.688+0000 7f4a97bff700 1 -- 172.21.15.145:0/3258864586 wait complete. 2023-08-25T04:44:45.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.689+0000 7f4a97bff700 1 Processor -- start 2023-08-25T04:44:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 -- start start 2023-08-25T04:44:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 0x7f4a901987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 0x7f4a9019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901a0370 con 0x7f4a90101b20 2023-08-25T04:44:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901a04b0 con 0x7f4a90100930 2023-08-25T04:44:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a97bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901a0640 con 0x7f4a901024c0 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a9519a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a9519a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54432/0 (socket says 172.21.15.145:54432) 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a9519a700 1 -- 172.21.15.145:0/2540161834 learned_addr learned my addr 172.21.15.145:0/2540161834 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a9619c700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 0x7f4a9019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.690+0000 7f4a9599b700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 0x7f4a901987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 -- 172.21.15.145:0/2540161834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 msgr2=0x7f4a9019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 0x7f4a9019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 -- 172.21.15.145:0/2540161834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 msgr2=0x7f4a901987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 0x7f4a901987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 -- 172.21.15.145:0/2540161834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0095f0 con 0x7f4a90101b20 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9599b700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 0x7f4a901987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a9519a700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80009940 con 0x7f4a90101b20 2023-08-25T04:44:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a80010460 con 0x7f4a90101b20 2023-08-25T04:44:45.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a901a0920 con 0x7f4a90101b20 2023-08-25T04:44:45.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000f620 con 0x7f4a90101b20 2023-08-25T04:44:45.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.691+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a901a0e80 con 0x7f4a90101b20 2023-08-25T04:44:45.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.692+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9010c140 con 0x7f4a90101b20 2023-08-25T04:44:45.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.694+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4a800108d0 con 0x7f4a90101b20 2023-08-25T04:44:45.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.696+0000 7f4a86ffd700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 0x7f4a7c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:45.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.696+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4a800cb370 con 0x7f4a90101b20 2023-08-25T04:44:45.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.696+0000 7f4a9599b700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 0x7f4a7c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:45.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.696+0000 7f4a9599b700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 0x7f4a7c0a24e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009990 tx=0x7f4a8c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:45.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.700+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4a80094da0 con 0x7f4a90101b20 2023-08-25T04:44:45.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.905+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a9010c300 con 0x7f4a90101b20 2023-08-25T04:44:45.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.907+0000 7f4a86ffd700 1 -- 172.21.15.145:0/2540161834 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f4a800983c0 con 0x7f4a90101b20 2023-08-25T04:44:45.909 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:45.909 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:45.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 msgr2=0x7f4a7c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 0x7f4a7c0a24e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009990 tx=0x7f4a8c01a040 comp rx=0 tx=0).stop 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 msgr2=0x7f4a90198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000dc60 comp rx=0 tx=0).stop 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 shutdown_connections 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4a7c0a0020 0x7f4a7c0a24e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a901024c0 0x7f4a9019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a90100930 0x7f4a901987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 --2- 172.21.15.145:0/2540161834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a90101b20 0x7f4a90198d30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:45.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 >> 172.21.15.145:0/2540161834 conn(0x7f4a900fbe60 msgr2=0x7f4a900fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:45.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 shutdown_connections 2023-08-25T04:44:45.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:45.910+0000 7f4a97bff700 1 -- 172.21.15.145:0/2540161834 wait complete. 2023-08-25T04:44:45.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:45.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:46.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 -- 172.21.15.145:0/4257548022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec073380 msgr2=0x7f10ec073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 --2- 172.21.15.145:0/4257548022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec073380 0x7f10ec073800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0098f0 tx=0x7f10dc009c00 comp rx=0 tx=0).stop 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 -- 172.21.15.145:0/4257548022 shutdown_connections 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 --2- 172.21.15.145:0/4257548022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec1087f0 0x7f10ec10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 --2- 172.21.15.145:0/4257548022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec074960 0x7f10ec072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.129+0000 7f10f39ff700 1 --2- 172.21.15.145:0/4257548022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec073380 0x7f10ec073800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.130+0000 7f10f39ff700 1 -- 172.21.15.145:0/4257548022 >> 172.21.15.145:0/4257548022 conn(0x7f10ec0fbea0 msgr2=0x7f10ec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:46.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.130+0000 7f10f39ff700 1 -- 172.21.15.145:0/4257548022 shutdown_connections 2023-08-25T04:44:46.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.130+0000 7f10f39ff700 1 -- 172.21.15.145:0/4257548022 wait complete. 2023-08-25T04:44:46.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 Processor -- start 2023-08-25T04:44:46.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 -- start start 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec074960 0x7f10ec198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec1087f0 0x7f10ec19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a03d0 con 0x7f10ec073380 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a0510 con 0x7f10ec1087f0 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f39ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a06a0 con 0x7f10ec074960 2023-08-25T04:44:46.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f179b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f179b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54456/0 (socket says 172.21.15.145:54456) 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f179b700 1 -- 172.21.15.145:0/1315286225 learned_addr learned my addr 172.21.15.145:0/1315286225 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f1f9c700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec1087f0 0x7f10ec19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.131+0000 7f10f0f9a700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec074960 0x7f10ec198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 -- 172.21.15.145:0/1315286225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec074960 msgr2=0x7f10ec198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec074960 0x7f10ec198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 -- 172.21.15.145:0/1315286225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec1087f0 msgr2=0x7f10ec19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec1087f0 0x7f10ec19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 -- 172.21.15.145:0/1315286225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80095f0 con 0x7f10ec073380 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f179b700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f10e800ec10 tx=0x7f10e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800cc90 con 0x7f10ec073380 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc009520 con 0x7f10ec073380 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e80044e0 con 0x7f10ec073380 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8010430 con 0x7f10ec073380 2023-08-25T04:44:46.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.132+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec1a0d00 con 0x7f10ec073380 2023-08-25T04:44:46.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.134+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec106ea0 con 0x7f10ec073380 2023-08-25T04:44:46.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.136+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f10e8003fb0 con 0x7f10ec073380 2023-08-25T04:44:46.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.137+0000 7f10e27fc700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 0x7f10d80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.137+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f10e8014070 con 0x7f10ec073380 2023-08-25T04:44:46.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.137+0000 7f10f0f9a700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 0x7f10d80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.138+0000 7f10f0f9a700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 0x7f10d80a2410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0098c0 tx=0x7f10dc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:46.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.141+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f10e80954e0 con 0x7f10ec073380 2023-08-25T04:44:46.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.341+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10ec02cd80 con 0x7f10ec073380 2023-08-25T04:44:46.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.342+0000 7f10e27fc700 1 -- 172.21.15.145:0/1315286225 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10e8098b00 con 0x7f10ec073380 2023-08-25T04:44:46.344 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:46.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 msgr2=0x7f10d80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 0x7f10d80a2410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0098c0 tx=0x7f10dc019040 comp rx=0 tx=0).stop 2023-08-25T04:44:46.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 msgr2=0x7f10ec198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f10e800ec10 tx=0x7f10e800ef20 comp rx=0 tx=0).stop 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 shutdown_connections 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f10d809ff50 0x7f10d80a2410 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10ec1087f0 0x7f10ec19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10ec073380 0x7f10ec198850 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 --2- 172.21.15.145:0/1315286225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec074960 0x7f10ec198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 >> 172.21.15.145:0/1315286225 conn(0x7f10ec0fbea0 msgr2=0x7f10ec0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:46.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 shutdown_connections 2023-08-25T04:44:46.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.344+0000 7f10f39ff700 1 -- 172.21.15.145:0/1315286225 wait complete. 2023-08-25T04:44:46.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.566+0000 7f4bb67ff700 1 -- 172.21.15.145:0/1273229450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 msgr2=0x7f4bb0073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.566+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/1273229450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb0073780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009990 tx=0x7f4ba4009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 -- 172.21.15.145:0/1273229450 shutdown_connections 2023-08-25T04:44:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/1273229450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb0108820 0x7f4bb010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/1273229450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb00748e0 0x7f4bb0072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/1273229450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb0073780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 -- 172.21.15.145:0/1273229450 >> 172.21.15.145:0/1273229450 conn(0x7f4bb00fbea0 msgr2=0x7f4bb00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 -- 172.21.15.145:0/1273229450 shutdown_connections 2023-08-25T04:44:46.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.567+0000 7f4bb67ff700 1 -- 172.21.15.145:0/1273229450 wait complete. 2023-08-25T04:44:46.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.568+0000 7f4bb67ff700 1 Processor -- start 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.568+0000 7f4bb67ff700 1 -- start start 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00748e0 0x7f4bb0198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 0x7f4bb019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01a0370 con 0x7f4bb0073300 2023-08-25T04:44:46.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01a04b0 con 0x7f4bb0108820 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb67ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01a0640 con 0x7f4bb00748e0 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4baffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4baffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54480/0 (socket says 172.21.15.145:54480) 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4baffff700 1 -- 172.21.15.145:0/150353778 learned_addr learned my addr 172.21.15.145:0/150353778 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4baf7fe700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00748e0 0x7f4bb0198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.569+0000 7f4bb4d9c700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 0x7f4bb019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 -- 172.21.15.145:0/150353778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00748e0 msgr2=0x7f4bb0198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00748e0 0x7f4bb0198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 -- 172.21.15.145:0/150353778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 msgr2=0x7f4bb019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 0x7f4bb019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 -- 172.21.15.145:0/150353778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009520 con 0x7f4bb0073300 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bb4d9c700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 0x7f4bb019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4baffff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000eb30 tx=0x7f4ba000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0009880 con 0x7f4bb0073300 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba40095f0 con 0x7f4bb0073300 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ba00044e0 con 0x7f4bb0073300 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0003e90 con 0x7f4bb0073300 2023-08-25T04:44:46.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.570+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb01a0cf0 con 0x7f4bb0073300 2023-08-25T04:44:46.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.571+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb0106f20 con 0x7f4bb0073300 2023-08-25T04:44:46.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.573+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4ba0004000 con 0x7f4bb0073300 2023-08-25T04:44:46.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.574+0000 7f4bad7fa700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 0x7f4b980a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:46.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.575+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4ba0014070 con 0x7f4bb0073300 2023-08-25T04:44:46.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.575+0000 7f4baf7fe700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 0x7f4b980a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:46.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.575+0000 7f4baf7fe700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 0x7f4b980a2410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400b680 tx=0x7f4ba4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:46.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.578+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4ba0095430 con 0x7f4bb0073300 2023-08-25T04:44:46.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.784+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bb002ccc0 con 0x7f4bb0073300 2023-08-25T04:44:46.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.784+0000 7f4bad7fa700 1 -- 172.21.15.145:0/150353778 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ba0098a50 con 0x7f4bb0073300 2023-08-25T04:44:46.786 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:46.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.786+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 msgr2=0x7f4b980a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.786+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 0x7f4b980a2410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400b680 tx=0x7f4ba4005fd0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.786+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 msgr2=0x7f4bb01987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.786+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000eb30 tx=0x7f4ba000ee40 comp rx=0 tx=0).stop 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 shutdown_connections 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b9809ff50 0x7f4b980a2410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb0108820 0x7f4bb019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb0073300 0x7f4bb01987f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 --2- 172.21.15.145:0/150353778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb00748e0 0x7f4bb0198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:46.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 >> 172.21.15.145:0/150353778 conn(0x7f4bb00fbea0 msgr2=0x7f4bb00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:46.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 shutdown_connections 2023-08-25T04:44:46.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:46.787+0000 7f4bb67ff700 1 -- 172.21.15.145:0/150353778 wait complete. 2023-08-25T04:44:46.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 -- 172.21.15.145:0/2165380489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c101b20 msgr2=0x7f4f9c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/2165380489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c101b20 0x7f4f9c101f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f900099c0 tx=0x7f4f90009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 -- 172.21.15.145:0/2165380489 shutdown_connections 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/2165380489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/2165380489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 0x7f4f9c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/2165380489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c101b20 0x7f4f9c101f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 -- 172.21.15.145:0/2165380489 >> 172.21.15.145:0/2165380489 conn(0x7f4f9c0fbea0 msgr2=0x7f4f9c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:47.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 -- 172.21.15.145:0/2165380489 shutdown_connections 2023-08-25T04:44:47.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.007+0000 7f4fa21ff700 1 -- 172.21.15.145:0/2165380489 wait complete. 2023-08-25T04:44:47.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 Processor -- start 2023-08-25T04:44:47.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 -- start start 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 0x7f4f9c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c101b20 0x7f4f9c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a0380 con 0x7f4f9c1024c0 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a04c0 con 0x7f4f9c100930 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4fa21ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1a0650 con 0x7f4f9c101b20 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4f9bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.008+0000 7f4f9bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54496/0 (socket says 172.21.15.145:54496) 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 -- 172.21.15.145:0/1595730050 learned_addr learned my addr 172.21.15.145:0/1595730050 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9b7fe700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 0x7f4f9c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9affd700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c101b20 0x7f4f9c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 -- 172.21.15.145:0/1595730050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c101b20 msgr2=0x7f4f9c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c101b20 0x7f4f9c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 -- 172.21.15.145:0/1595730050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 msgr2=0x7f4f9c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 0x7f4f9c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 -- 172.21.15.145:0/1595730050 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900095f0 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f9bfff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007ee0 tx=0x7f4f9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94017070 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9400f040 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c1a0930 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.009+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94005580 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.010+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1a0df0 con 0x7f4f9c1024c0 2023-08-25T04:44:47.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.010+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c10c140 con 0x7f4f9c1024c0 2023-08-25T04:44:47.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.012+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4f9400db20 con 0x7f4f9c1024c0 2023-08-25T04:44:47.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.013+0000 7f4f98ff9700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 0x7f4f840a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.014+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4f940caca0 con 0x7f4f9c1024c0 2023-08-25T04:44:47.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.014+0000 7f4f9b7fe700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 0x7f4f840a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.015+0000 7f4f9b7fe700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 0x7f4f840a2410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c101980 tx=0x7f4f8c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.018+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4f940946d0 con 0x7f4f9c1024c0 2023-08-25T04:44:47.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:46 smithi145 ceph-mon[97681]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-08-25T04:44:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2540161834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1315286225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/150353778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:46 smithi162 ceph-mon[130941]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-08-25T04:44:47.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2540161834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:47.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1315286225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/150353778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:46 smithi183 ceph-mon[131414]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-08-25T04:44:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2540161834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1315286225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/150353778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:47.216 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.215+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f9c1a0f80 con 0x7f4f9c1024c0 2023-08-25T04:44:47.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.215+0000 7f4f98ff9700 1 -- 172.21.15.145:0/1595730050 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f94097cf0 con 0x7f4f9c1024c0 2023-08-25T04:44:47.217 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:47.219 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 msgr2=0x7f4f840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.219 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 0x7f4f840a2410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c101980 tx=0x7f4f8c007400 comp rx=0 tx=0).stop 2023-08-25T04:44:47.219 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 msgr2=0x7f4f9c19fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007ee0 tx=0x7f4f9400d3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 shutdown_connections 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4f8409ff50 0x7f4f840a2410 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f9c1024c0 0x7f4f9c19fdb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f9c100930 0x7f4f9c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 --2- 172.21.15.145:0/1595730050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f9c101b20 0x7f4f9c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.220 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 >> 172.21.15.145:0/1595730050 conn(0x7f4f9c0fbea0 msgr2=0x7f4f9c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:47.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 shutdown_connections 2023-08-25T04:44:47.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.218+0000 7f4fa21ff700 1 -- 172.21.15.145:0/1595730050 wait complete. 2023-08-25T04:44:47.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:47.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.437+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/2748384636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 msgr2=0x7f9b3810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.437+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/2748384636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3810ac70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009f90 tx=0x7f9b3000b330 comp rx=0 tx=0).stop 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/2748384636 shutdown_connections 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/2748384636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3810ac70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/2748384636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/2748384636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b380737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/2748384636 >> 172.21.15.145:0/2748384636 conn(0x7f9b380fbea0 msgr2=0x7f9b380fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:47.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/2748384636 shutdown_connections 2023-08-25T04:44:47.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.438+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/2748384636 wait complete. 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 Processor -- start 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 -- start start 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b381988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a0470 con 0x7f9b38074900 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a05b0 con 0x7f9b38108800 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a0740 con 0x7f9b38073320 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b367fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b367fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54504/0 (socket says 172.21.15.145:54504) 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b36ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b381988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b367fc700 1 -- 172.21.15.145:0/1861926713 learned_addr learned my addr 172.21.15.145:0/1861926713 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.439+0000 7f9b377fe700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 -- 172.21.15.145:0/1861926713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 msgr2=0x7f9b381988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b381988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 -- 172.21.15.145:0/1861926713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 msgr2=0x7f9b3819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 -- 172.21.15.145:0/1861926713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280095f0 con 0x7f9b38074900 2023-08-25T04:44:47.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b36ffd700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b381988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b367fc700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00e9a0 tx=0x7f9b2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c009790 con 0x7f9b38074900 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b30009c40 con 0x7f9b38074900 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c010430 con 0x7f9b38074900 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c018680 con 0x7f9b38074900 2023-08-25T04:44:47.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.440+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b381a0df0 con 0x7f9b38074900 2023-08-25T04:44:47.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.441+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38106f00 con 0x7f9b38074900 2023-08-25T04:44:47.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.444+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f9b2c0105a0 con 0x7f9b38074900 2023-08-25T04:44:47.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.445+0000 7f9b1ffff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 0x7f9b200a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.445+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9b2c014070 con 0x7f9b38074900 2023-08-25T04:44:47.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.445+0000 7f9b36ffd700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 0x7f9b200a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.446+0000 7f9b36ffd700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 0x7f9b200a2410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005fd0 tx=0x7f9b28009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.447+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9b2c095340 con 0x7f9b38074900 2023-08-25T04:44:47.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.649+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9b3810e360 con 0x7f9b38074900 2023-08-25T04:44:47.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.650+0000 7f9b1ffff700 1 -- 172.21.15.145:0/1861926713 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9b2c026020 con 0x7f9b38074900 2023-08-25T04:44:47.652 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:47.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 msgr2=0x7f9b200a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 0x7f9b200a2410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005fd0 tx=0x7f9b28009310 comp rx=0 tx=0).stop 2023-08-25T04:44:47.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 msgr2=0x7f9b38198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00e9a0 tx=0x7f9b2c00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 shutdown_connections 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f9b2009ff50 0x7f9b200a2410 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b38108800 0x7f9b3819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b38073320 0x7f9b381988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 --2- 172.21.15.145:0/1861926713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38074900 0x7f9b38198e30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 >> 172.21.15.145:0/1861926713 conn(0x7f9b380fbea0 msgr2=0x7f9b380fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:47.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 shutdown_connections 2023-08-25T04:44:47.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.652+0000 7f9b3d5ff700 1 -- 172.21.15.145:0/1861926713 wait complete. 2023-08-25T04:44:47.672 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:47.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 -- 172.21.15.145:0/1955253064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 msgr2=0x7f8960108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 --2- 172.21.15.145:0/1955253064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f8960108d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8958009fc0 tx=0x7f895800b330 comp rx=0 tx=0).stop 2023-08-25T04:44:47.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 -- 172.21.15.145:0/1955253064 shutdown_connections 2023-08-25T04:44:47.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 --2- 172.21.15.145:0/1955253064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f8960108d70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 --2- 172.21.15.145:0/1955253064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960100930 0x7f8960100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 --2- 172.21.15.145:0/1955253064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960101b20 0x7f8960101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 -- 172.21.15.145:0/1955253064 >> 172.21.15.145:0/1955253064 conn(0x7f89600fbe60 msgr2=0x7f89600fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:47.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 -- 172.21.15.145:0/1955253064 shutdown_connections 2023-08-25T04:44:47.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.870+0000 7f8965dff700 1 -- 172.21.15.145:0/1955253064 wait complete. 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 Processor -- start 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 -- start start 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960101b20 0x7f8960198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f896019fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a0460 con 0x7f89601024c0 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a05a0 con 0x7f8960100930 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.871+0000 7f8965dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a0730 con 0x7f8960101b20 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60776/0 (socket says 172.21.15.145:60776) 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f896019fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 -- 172.21.15.145:0/4000577343 learned_addr learned my addr 172.21.15.145:0/4000577343 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f8957fff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960101b20 0x7f8960198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 -- 172.21.15.145:0/4000577343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960101b20 msgr2=0x7f8960198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960101b20 0x7f8960198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 -- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 msgr2=0x7f896019fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f896019fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 -- 172.21.15.145:0/4000577343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89480095f0 con 0x7f8960100930 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895ffff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f896019fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.872+0000 7f895f7fe700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f895000d980 tx=0x7f895000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.873+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950004020 con 0x7f8960100930 2023-08-25T04:44:47.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.873+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8950009730 con 0x7f8960100930 2023-08-25T04:44:47.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.873+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8958009c40 con 0x7f8960100930 2023-08-25T04:44:47.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.873+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003a30 con 0x7f8960100930 2023-08-25T04:44:47.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.873+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89601a0d90 con 0x7f8960100930 2023-08-25T04:44:47.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.874+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896010c1a0 con 0x7f8960100930 2023-08-25T04:44:47.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.876+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f89500098a0 con 0x7f8960100930 2023-08-25T04:44:47.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.877+0000 7f895d7fa700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 0x7f894c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:47.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.877+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f89500cb370 con 0x7f8960100930 2023-08-25T04:44:47.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.877+0000 7f8957fff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 0x7f894c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:47.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.878+0000 7f8957fff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 0x7f894c0a2460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8948005fd0 tx=0x7f8948009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:47.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:47.879+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8950094e00 con 0x7f8960100930 2023-08-25T04:44:48.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:47 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1595730050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:47 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1861926713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.085+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f896010c3b0 con 0x7f8960100930 2023-08-25T04:44:48.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.087+0000 7f895d7fa700 1 -- 172.21.15.145:0/4000577343 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f8950098420 con 0x7f8960100930 2023-08-25T04:44:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:48.089 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 msgr2=0x7f894c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 0x7f894c0a2460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8948005fd0 tx=0x7f8948009310 comp rx=0 tx=0).stop 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 msgr2=0x7f89601988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f895000d980 tx=0x7f895000dc90 comp rx=0 tx=0).stop 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 shutdown_connections 2023-08-25T04:44:48.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f894c09ffa0 0x7f894c0a2460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601024c0 0x7f896019fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960100930 0x7f89601988e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 --2- 172.21.15.145:0/4000577343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8960101b20 0x7f8960198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 >> 172.21.15.145:0/4000577343 conn(0x7f89600fbe60 msgr2=0x7f8960107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 shutdown_connections 2023-08-25T04:44:48.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.090+0000 7f8965dff700 1 -- 172.21.15.145:0/4000577343 wait complete. 2023-08-25T04:44:48.111 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:48.112 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:47 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1595730050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:47 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1861926713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:47 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1595730050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:47 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1861926713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:48.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 -- 172.21.15.145:0/3965154567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4100930 msgr2=0x7fcff4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 --2- 172.21.15.145:0/3965154567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4100930 0x7fcff4100d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009920 tx=0x7fcfe4009c30 comp rx=0 tx=0).stop 2023-08-25T04:44:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 -- 172.21.15.145:0/3965154567 shutdown_connections 2023-08-25T04:44:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 --2- 172.21.15.145:0/3965154567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 --2- 172.21.15.145:0/3965154567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4100930 0x7fcff4100d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 --2- 172.21.15.145:0/3965154567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4101b20 0x7fcff4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 -- 172.21.15.145:0/3965154567 >> 172.21.15.145:0/3965154567 conn(0x7fcff40fbea0 msgr2=0x7fcff40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:48.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 -- 172.21.15.145:0/3965154567 shutdown_connections 2023-08-25T04:44:48.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.310+0000 7fcff8bff700 1 -- 172.21.15.145:0/3965154567 wait complete. 2023-08-25T04:44:48.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 Processor -- start 2023-08-25T04:44:48.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 -- start start 2023-08-25T04:44:48.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 0x7fcff40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4101b20 0x7fcff4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40739f0 con 0x7fcff41024c0 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4073b60 con 0x7fcff4101b20 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.311+0000 7fcff8bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41a07f0 con 0x7fcff4100930 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff259c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 0x7fcff40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54536/0 (socket says 172.21.15.145:54536) 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 -- 172.21.15.145:0/2279462231 learned_addr learned my addr 172.21.15.145:0/2279462231 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcfebfff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4101b20 0x7fcff4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 -- 172.21.15.145:0/2279462231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 msgr2=0x7fcff40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 0x7fcff40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 -- 172.21.15.145:0/2279462231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4101b20 msgr2=0x7fcff4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4101b20 0x7fcff4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 -- 172.21.15.145:0/2279462231 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009d00 con 0x7fcff41024c0 2023-08-25T04:44:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff259c700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 0x7fcff40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.312+0000 7fcff2d9d700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00e3c0 tx=0x7fcfec00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.313+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0090d0 con 0x7fcff41024c0 2023-08-25T04:44:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.313+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4009520 con 0x7fcff41024c0 2023-08-25T04:44:48.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.313+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfec00f040 con 0x7fcff41024c0 2023-08-25T04:44:48.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.313+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec007c20 con 0x7fcff41024c0 2023-08-25T04:44:48.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.313+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff41a0e50 con 0x7fcff41024c0 2023-08-25T04:44:48.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.314+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff4107ff0 con 0x7fcff41024c0 2023-08-25T04:44:48.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.316+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcfec003a00 con 0x7fcff41024c0 2023-08-25T04:44:48.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.317+0000 7fcfeb7fe700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 0x7fcfe00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.317+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcfec0cb2b0 con 0x7fcff41024c0 2023-08-25T04:44:48.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.317+0000 7fcff259c700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 0x7fcfe00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.318+0000 7fcff259c700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 0x7fcfe00a2410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40098f0 tx=0x7fcfe4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:48.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.321+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcfec094d70 con 0x7fcff41024c0 2023-08-25T04:44:48.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.518+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcff402cd00 con 0x7fcff41024c0 2023-08-25T04:44:48.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.519+0000 7fcfeb7fe700 1 -- 172.21.15.145:0/2279462231 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfec098390 con 0x7fcff41024c0 2023-08-25T04:44:48.521 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:48.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 msgr2=0x7fcfe00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 0x7fcfe00a2410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40098f0 tx=0x7fcfe4019040 comp rx=0 tx=0).stop 2023-08-25T04:44:48.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 msgr2=0x7fcff4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00e3c0 tx=0x7fcfec00e6d0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 shutdown_connections 2023-08-25T04:44:48.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfe009ff50 0x7fcfe00a2410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcff41024c0 0x7fcff4073250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcff4100930 0x7fcff40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 --2- 172.21.15.145:0/2279462231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcff4101b20 0x7fcff4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 >> 172.21.15.145:0/2279462231 conn(0x7fcff40fbea0 msgr2=0x7fcff4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:48.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 shutdown_connections 2023-08-25T04:44:48.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.521+0000 7fcff8bff700 1 -- 172.21.15.145:0/2279462231 wait complete. 2023-08-25T04:44:48.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:48.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/1864194698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a41087f0 msgr2=0x7fa1a410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/1864194698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a41087f0 0x7fa1a410ac80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009f90 tx=0x7fa19c00b330 comp rx=0 tx=0).stop 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/1864194698 shutdown_connections 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/1864194698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a41087f0 0x7fa1a410ac80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/1864194698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 0x7fa1a4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/1864194698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a4073380 0x7fa1a4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/1864194698 >> 172.21.15.145:0/1864194698 conn(0x7fa1a40fbea0 msgr2=0x7fa1a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/1864194698 shutdown_connections 2023-08-25T04:44:48.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.744+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/1864194698 wait complete. 2023-08-25T04:44:48.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 Processor -- start 2023-08-25T04:44:48.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 -- start start 2023-08-25T04:44:48.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 0x7fa1a4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a41087f0 0x7fa1a419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41a0570 con 0x7fa1a4073380 2023-08-25T04:44:48.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41a06b0 con 0x7fa1a4074960 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.745+0000 7fa1ac1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41a0840 con 0x7fa1a41087f0 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54548/0 (socket says 172.21.15.145:54548) 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 -- 172.21.15.145:0/2847346455 learned_addr learned my addr 172.21.15.145:0/2847346455 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1aa79c700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a41087f0 0x7fa1a419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a979a700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 0x7fa1a4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 -- 172.21.15.145:0/2847346455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a41087f0 msgr2=0x7fa1a419ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a41087f0 0x7fa1a419ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 -- 172.21.15.145:0/2847346455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 msgr2=0x7fa1a4198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 0x7fa1a4198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 -- 172.21.15.145:0/2847346455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c009c40 con 0x7fa1a4073380 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa1a9f9b700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000e9c0 tx=0x7fa1a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:48.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.746+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004a30 con 0x7fa1a4073380 2023-08-25T04:44:48.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.747+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a41a0b20 con 0x7fa1a4073380 2023-08-25T04:44:48.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.747+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a000cc80 con 0x7fa1a4073380 2023-08-25T04:44:48.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.747+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00054e0 con 0x7fa1a4073380 2023-08-25T04:44:48.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.747+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a41a1030 con 0x7fa1a4073380 2023-08-25T04:44:48.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.748+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a4106ea0 con 0x7fa1a4073380 2023-08-25T04:44:48.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.749+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fa1a0004b90 con 0x7fa1a4073380 2023-08-25T04:44:48.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.751+0000 7fa19affd700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 0x7fa1900a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:48.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.751+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa1a0014070 con 0x7fa1a4073380 2023-08-25T04:44:48.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.751+0000 7fa1a979a700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 0x7fa1900a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:48.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.752+0000 7fa1a979a700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 0x7fa1900a2410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4074680 tx=0x7fa194009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:48.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.754+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa1a0094fd0 con 0x7fa1a4073380 2023-08-25T04:44:48.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.959+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1a402cc20 con 0x7fa1a4073380 2023-08-25T04:44:48.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.959+0000 7fa19affd700 1 -- 172.21.15.145:0/2847346455 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1a00985f0 con 0x7fa1a4073380 2023-08-25T04:44:48.961 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:48.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 msgr2=0x7fa1900a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 0x7fa1900a2410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4074680 tx=0x7fa194009260 comp rx=0 tx=0).stop 2023-08-25T04:44:48.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 msgr2=0x7fa1a41989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:48.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000e9c0 tx=0x7fa1a000ecd0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 shutdown_connections 2023-08-25T04:44:48.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa19009ff50 0x7fa1900a2410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1a41087f0 0x7fa1a419ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1a4073380 0x7fa1a41989f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 --2- 172.21.15.145:0/2847346455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1a4074960 0x7fa1a4198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:48.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 >> 172.21.15.145:0/2847346455 conn(0x7fa1a40fbea0 msgr2=0x7fa1a40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:48.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 shutdown_connections 2023-08-25T04:44:48.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:48.962+0000 7fa1ac1ff700 1 -- 172.21.15.145:0/2847346455 wait complete. 2023-08-25T04:44:48.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:49.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:48 smithi145 ceph-mon[97681]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:49.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:48 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4000577343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:49.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:48 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2279462231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:49.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:48 smithi162 ceph-mon[130941]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:49.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:48 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4000577343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:49.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:48 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2279462231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:49.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:48 smithi183 ceph-mon[131414]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:44:49.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:48 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4000577343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:49.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:48 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2279462231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:49.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 -- 172.21.15.145:0/3490348984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 msgr2=0x7fbaa8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/3490348984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8101f60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0099c0 tx=0x7fba9c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 -- 172.21.15.145:0/3490348984 shutdown_connections 2023-08-25T04:44:49.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/3490348984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa81024a0 0x7fbaa8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/3490348984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa8100910 0x7fbaa8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/3490348984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8101f60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 -- 172.21.15.145:0/3490348984 >> 172.21.15.145:0/3490348984 conn(0x7fbaa80fbe60 msgr2=0x7fbaa80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:49.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 -- 172.21.15.145:0/3490348984 shutdown_connections 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.183+0000 7fbaad1ff700 1 -- 172.21.15.145:0/3490348984 wait complete. 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.184+0000 7fbaad1ff700 1 Processor -- start 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.184+0000 7fbaad1ff700 1 -- start start 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa8100910 0x7fbaa8196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa81024a0 0x7fbaa819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa819e1e0 con 0x7fbaa8101b00 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa819e320 con 0x7fbaa8100910 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaad1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa819e4b0 con 0x7fbaa81024a0 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54566/0 (socket says 172.21.15.145:54566) 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 -- 172.21.15.145:0/2924037170 learned_addr learned my addr 172.21.15.145:0/2924037170 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa6d9d700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa8100910 0x7fbaa8196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa759e700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa81024a0 0x7fbaa819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 -- 172.21.15.145:0/2924037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa81024a0 msgr2=0x7fbaa819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa81024a0 0x7fbaa819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 -- 172.21.15.145:0/2924037170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa8100910 msgr2=0x7fbaa8196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa8100910 0x7fbaa8196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.185+0000 7fbaa659c700 1 -- 172.21.15.145:0/2924037170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c0095f0 con 0x7fbaa8101b00 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fbaa659c700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bf40 tx=0x7fba9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c01d070 con 0x7fbaa8101b00 2023-08-25T04:44:49.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa819e730 con 0x7fbaa8101b00 2023-08-25T04:44:49.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba9c005510 con 0x7fbaa8101b00 2023-08-25T04:44:49.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c004c00 con 0x7fbaa8101b00 2023-08-25T04:44:49.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.186+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa819ebb0 con 0x7fbaa8101b00 2023-08-25T04:44:49.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.187+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa810c170 con 0x7fbaa8101b00 2023-08-25T04:44:49.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.189+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fba9c00fb80 con 0x7fbaa8101b00 2023-08-25T04:44:49.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.190+0000 7fba8ffff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 0x7fba900a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.190+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fba9c0cc7f0 con 0x7fbaa8101b00 2023-08-25T04:44:49.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.191+0000 7fbaa6d9d700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 0x7fba900a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.191+0000 7fbaa6d9d700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 0x7fba900a2410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8101960 tx=0x7fba98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.193+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fba9c096310 con 0x7fbaa8101b00 2023-08-25T04:44:49.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.394+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbaa8107890 con 0x7fbaa8101b00 2023-08-25T04:44:49.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.395+0000 7fba8ffff700 1 -- 172.21.15.145:0/2924037170 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba9c099930 con 0x7fbaa8101b00 2023-08-25T04:44:49.401 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:49.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 msgr2=0x7fba900a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 0x7fba900a2410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8101960 tx=0x7fba98009260 comp rx=0 tx=0).stop 2023-08-25T04:44:49.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 msgr2=0x7fbaa8196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00bf40 tx=0x7fba9c00f690 comp rx=0 tx=0).stop 2023-08-25T04:44:49.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 shutdown_connections 2023-08-25T04:44:49.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fba9009ff50 0x7fba900a2410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaa81024a0 0x7fbaa819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbaa8100910 0x7fbaa8196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 --2- 172.21.15.145:0/2924037170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa8101b00 0x7fbaa8196ba0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 >> 172.21.15.145:0/2924037170 conn(0x7fbaa80fbe60 msgr2=0x7fbaa80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:49.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 shutdown_connections 2023-08-25T04:44:49.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.397+0000 7fbaad1ff700 1 -- 172.21.15.145:0/2924037170 wait complete. 2023-08-25T04:44:49.418 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:49.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 -- 172.21.15.145:0/3906534216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc1024a0 msgr2=0x7f8acc108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/3906534216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc1024a0 0x7f8acc108d50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400b0d0 tx=0x7f8ac400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 -- 172.21.15.145:0/3906534216 shutdown_connections 2023-08-25T04:44:49.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/3906534216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc1024a0 0x7f8acc108d50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/3906534216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 0x7f8acc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/3906534216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc101b00 0x7f8acc101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 -- 172.21.15.145:0/3906534216 >> 172.21.15.145:0/3906534216 conn(0x7f8acc0fbe60 msgr2=0x7f8acc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:49.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 -- 172.21.15.145:0/3906534216 shutdown_connections 2023-08-25T04:44:49.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.622+0000 7f8ad35ff700 1 -- 172.21.15.145:0/3906534216 wait complete. 2023-08-25T04:44:49.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.623+0000 7f8ad35ff700 1 Processor -- start 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.623+0000 7f8ad35ff700 1 -- start start 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 0x7f8acc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc1024a0 0x7f8acc19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc1a0550 con 0x7f8acc101b00 2023-08-25T04:44:49.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc1a0690 con 0x7f8acc100910 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad35ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc1a0820 con 0x7f8acc1024a0 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54598/0 (socket says 172.21.15.145:54598) 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 -- 172.21.15.145:0/1830292707 learned_addr learned my addr 172.21.15.145:0/1830292707 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad139b700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 0x7f8acc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad1b9c700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc1024a0 0x7f8acc19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 -- 172.21.15.145:0/1830292707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc1024a0 msgr2=0x7f8acc19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc1024a0 0x7f8acc19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 -- 172.21.15.145:0/1830292707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 msgr2=0x7f8acc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 0x7f8acc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.624+0000 7f8ad0b9a700 1 -- 172.21.15.145:0/1830292707 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009d00 con 0x7f8acc101b00 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ad0b9a700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00e9a0 tx=0x7f8abc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc009790 con 0x7f8acc101b00 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc1a0b00 con 0x7f8acc101b00 2023-08-25T04:44:49.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8abc010430 con 0x7f8acc101b00 2023-08-25T04:44:49.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc018680 con 0x7f8acc101b00 2023-08-25T04:44:49.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.625+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc1a1010 con 0x7f8acc101b00 2023-08-25T04:44:49.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.626+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc192f70 con 0x7f8acc101b00 2023-08-25T04:44:49.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.628+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f8abc0105a0 con 0x7f8acc101b00 2023-08-25T04:44:49.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.629+0000 7f8ac27fc700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 0x7f8ab80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:49.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.629+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8abc014070 con 0x7f8acc101b00 2023-08-25T04:44:49.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.629+0000 7f8ad139b700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 0x7f8ab80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:49.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.630+0000 7f8ad139b700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 0x7f8ab80a2410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8acc101960 tx=0x7f8ac8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:49.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.633+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8abc051e00 con 0x7f8acc101b00 2023-08-25T04:44:49.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.839+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8acc02cde0 con 0x7f8acc101b00 2023-08-25T04:44:49.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.839+0000 7f8ac27fc700 1 -- 172.21.15.145:0/1830292707 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8abc095090 con 0x7f8acc101b00 2023-08-25T04:44:49.842 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:49.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 msgr2=0x7f8ab80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 0x7f8ab80a2410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8acc101960 tx=0x7f8ac8009260 comp rx=0 tx=0).stop 2023-08-25T04:44:49.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 msgr2=0x7f8acc198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:49.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00e9a0 tx=0x7f8abc00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 shutdown_connections 2023-08-25T04:44:49.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f8ab809ff50 0x7f8ab80a2410 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8acc1024a0 0x7f8acc19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8acc100910 0x7f8acc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 --2- 172.21.15.145:0/1830292707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8acc101b00 0x7f8acc198f10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:49.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 >> 172.21.15.145:0/1830292707 conn(0x7f8acc0fbe60 msgr2=0x7f8acc1073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:49.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 shutdown_connections 2023-08-25T04:44:49.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:49.842+0000 7f8ad35ff700 1 -- 172.21.15.145:0/1830292707 wait complete. 2023-08-25T04:44:49.862 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-08-25T04:44:49.862 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:50.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.057+0000 7fd3f53ff700 1 -- 172.21.15.145:0/387345905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f0101b20 msgr2=0x7fd3f0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.057+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/387345905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f0101b20 0x7fd3f0101f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098f0 tx=0x7fd3e4009c00 comp rx=0 tx=0).stop 2023-08-25T04:44:50.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 -- 172.21.15.145:0/387345905 shutdown_connections 2023-08-25T04:44:50.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/387345905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f01024c0 0x7fd3f0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/387345905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 0x7fd3f0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/387345905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f0101b20 0x7fd3f0101f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 -- 172.21.15.145:0/387345905 >> 172.21.15.145:0/387345905 conn(0x7fd3f00fbe60 msgr2=0x7fd3f00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:50.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 -- 172.21.15.145:0/387345905 shutdown_connections 2023-08-25T04:44:50.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.058+0000 7fd3f53ff700 1 -- 172.21.15.145:0/387345905 wait complete. 2023-08-25T04:44:50.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 Processor -- start 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 -- start start 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 0x7fd3f0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f01024c0 0x7fd3f019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a03d0 con 0x7fd3f0101b20 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a0510 con 0x7fd3f0100930 2023-08-25T04:44:50.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3f53ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a06a0 con 0x7fd3f01024c0 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3ee7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3ee7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54620/0 (socket says 172.21.15.145:54620) 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3ee7fc700 1 -- 172.21.15.145:0/3945445202 learned_addr learned my addr 172.21.15.145:0/3945445202 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.059+0000 7fd3ef7fe700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f01024c0 0x7fd3f019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3eeffd700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 0x7fd3f0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 -- 172.21.15.145:0/3945445202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f01024c0 msgr2=0x7fd3f019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f01024c0 0x7fd3f019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 -- 172.21.15.145:0/3945445202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 msgr2=0x7fd3f0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 0x7fd3f0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 -- 172.21.15.145:0/3945445202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e4009520 con 0x7fd3f0101b20 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3ee7fc700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e401c070 con 0x7fd3f0101b20 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e400bc10 con 0x7fd3f0101b20 2023-08-25T04:44:50.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f01a0920 con 0x7fd3f0101b20 2023-08-25T04:44:50.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.060+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4004a20 con 0x7fd3f0101b20 2023-08-25T04:44:50.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.061+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f01a0d40 con 0x7fd3f0101b20 2023-08-25T04:44:50.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.062+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f0192f80 con 0x7fd3f0101b20 2023-08-25T04:44:50.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.063+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd3e4020070 con 0x7fd3f0101b20 2023-08-25T04:44:50.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.065+0000 7fd3d7fff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 0x7fd3d80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.065+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd3e40cc1b0 con 0x7fd3f0101b20 2023-08-25T04:44:50.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.065+0000 7fd3eeffd700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 0x7fd3d80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.065+0000 7fd3eeffd700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 0x7fd3d80a2460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0101980 tx=0x7fd3e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:50.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.068+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd3e405ce00 con 0x7fd3f0101b20 2023-08-25T04:44:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:49 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2847346455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:49 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2924037170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:49 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2847346455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:49 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2924037170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:49 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2847346455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:49 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2924037170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:50.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.310+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3f010c190 con 0x7fd3f0101b20 2023-08-25T04:44:50.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.311+0000 7fd3d7fff700 1 -- 172.21.15.145:0/3945445202 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7fd3e4095cd0 con 0x7fd3f0101b20 2023-08-25T04:44:50.313 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:50.313 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 msgr2=0x7fd3d80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 0x7fd3d80a2460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0101980 tx=0x7fd3e0009260 comp rx=0 tx=0).stop 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 msgr2=0x7fd3f0198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e400b560 comp rx=0 tx=0).stop 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 shutdown_connections 2023-08-25T04:44:50.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd3d809ffa0 0x7fd3d80a2460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3f01024c0 0x7fd3f019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3f0100930 0x7fd3f0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 --2- 172.21.15.145:0/3945445202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3f0101b20 0x7fd3f0198d90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 >> 172.21.15.145:0/3945445202 conn(0x7fd3f00fbe60 msgr2=0x7fd3f0107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 shutdown_connections 2023-08-25T04:44:50.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.313+0000 7fd3f53ff700 1 -- 172.21.15.145:0/3945445202 wait complete. 2023-08-25T04:44:50.317 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:50.336 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-08-25T04:44:50.336 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-08-25T04:44:50.336 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T04:44:50.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 -- 172.21.15.145:0/368313336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf84103d10 msgr2=0x7fcf84106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.552 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 --2- 172.21.15.145:0/368313336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf84103d10 0x7fcf84106100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009990 tx=0x7fcf6c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 -- 172.21.15.145:0/368313336 shutdown_connections 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 --2- 172.21.15.145:0/368313336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 0x7fcf84108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 --2- 172.21.15.145:0/368313336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf841013e0 0x7fcf841037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 --2- 172.21.15.145:0/368313336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf84103d10 0x7fcf84106100 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 -- 172.21.15.145:0/368313336 >> 172.21.15.145:0/368313336 conn(0x7fcf840fad50 msgr2=0x7fcf840fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.550+0000 7fcf88bff700 1 -- 172.21.15.145:0/368313336 shutdown_connections 2023-08-25T04:44:50.553 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.551+0000 7fcf88bff700 1 -- 172.21.15.145:0/368313336 wait complete. 2023-08-25T04:44:50.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.551+0000 7fcf88bff700 1 Processor -- start 2023-08-25T04:44:50.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 -- start start 2023-08-25T04:44:50.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf84103d10 0x7fcf84196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 0x7fcf8419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419e150 con 0x7fcf841013e0 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419e290 con 0x7fcf84103d10 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf88bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419e420 con 0x7fcf84106640 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf8259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf8259c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54630/0 (socket says 172.21.15.145:54630) 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf8259c700 1 -- 172.21.15.145:0/871442187 learned_addr learned my addr 172.21.15.145:0/871442187 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf7bfff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf84103d10 0x7fcf84196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.552+0000 7fcf82d9d700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 0x7fcf8419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 -- 172.21.15.145:0/871442187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 msgr2=0x7fcf8419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 0x7fcf8419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 -- 172.21.15.145:0/871442187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf84103d10 msgr2=0x7fcf84196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf84103d10 0x7fcf84196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 -- 172.21.15.145:0/871442187 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0095f0 con 0x7fcf841013e0 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf8259c700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400ef10 tx=0x7fcf7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400cad0 con 0x7fcf841013e0 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf7400f040 con 0x7fcf841013e0 2023-08-25T04:44:50.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74007ea0 con 0x7fcf841013e0 2023-08-25T04:44:50.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8419e6a0 con 0x7fcf841013e0 2023-08-25T04:44:50.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.553+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8419eb60 con 0x7fcf841013e0 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.554+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8410c140 con 0x7fcf841013e0 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.556+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcf74007500 con 0x7fcf841013e0 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.557+0000 7fcf7b7fe700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 0x7fcf700a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.558+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcf740cb930 con 0x7fcf841013e0 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.558+0000 7fcf7bfff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 0x7fcf700a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:50.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.558+0000 7fcf7bfff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 0x7fcf700a24e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b2e0 tx=0x7fcf6c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:50.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.562+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcf740953c0 con 0x7fcf841013e0 2023-08-25T04:44:50.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.759+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf84102fb0 con 0x7fcf700a0020 2023-08-25T04:44:50.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.772+0000 7fcf7b7fe700 1 -- 172.21.15.145:0/871442187 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427232 (secure 0 0 0) 0x7fcf84102fb0 con 0x7fcf700a0020 2023-08-25T04:44:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:50.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.777+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 msgr2=0x7fcf700a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.777+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 0x7fcf700a24e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b2e0 tx=0x7fcf6c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.777+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 msgr2=0x7fcf841965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:50.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.777+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400ef10 tx=0x7fcf7400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 shutdown_connections 2023-08-25T04:44:50.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcf700a0020 0x7fcf700a24e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf84106640 0x7fcf8419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf841013e0 0x7fcf841965d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 --2- 172.21.15.145:0/871442187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf84103d10 0x7fcf84196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 >> 172.21.15.145:0/871442187 conn(0x7fcf840fad50 msgr2=0x7fcf841028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 shutdown_connections 2023-08-25T04:44:50.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:50.778+0000 7fcf88bff700 1 -- 172.21.15.145:0/871442187 wait complete. 2023-08-25T04:44:50.788 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:44:50.798 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-08-25T04:44:49.714433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":712,"num_read_kb":421,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":11652,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":11931648,"data_stored":7586071,"data_compressed":318375,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002989"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765465+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:43:34.765465+0000","last_peered":"2023-08-25T04:43:34.765465+0000","last_clean":"2023-08-25T04:43:34.765465+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:43:34.765465+0000","last_undegraded":"2023-08-25T04:43:34.765465+0000","last_fullsized":"2023-08-25T04:43:34.765465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:59.835651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764768+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:43:34.764768+0000","last_peered":"2023-08-25T04:43:34.764768+0000","last_clean":"2023-08-25T04:43:34.764768+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:43:34.764768+0000","last_undegraded":"2023-08-25T04:43:34.764768+0000","last_fullsized":"2023-08-25T04:43:34.764768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:56:09.848052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766305+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:43:34.766305+0000","last_peered":"2023-08-25T04:43:34.766305+0000","last_clean":"2023-08-25T04:43:34.766305+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:43:34.766305+0000","last_undegraded":"2023-08-25T04:43:34.766305+0000","last_fullsized":"2023-08-25T04:43:34.766305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:56.683325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762343+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:43:34.762343+0000","last_peered":"2023-08-25T04:43:34.762343+0000","last_clean":"2023-08-25T04:43:34.762343+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:43:34.762343+0000","last_undegraded":"2023-08-25T04:43:34.762343+0000","last_fullsized":"2023-08-25T04:43:34.762343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:57:35.343381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766096+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:43:34.766096+0000","last_peered":"2023-08-25T04:43:34.766096+0000","last_clean":"2023-08-25T04:43:34.766096+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:43:34.766096+0000","last_undegraded":"2023-08-25T04:43:34.766096+0000","last_fullsized":"2023-08-25T04:43:34.766096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:53:26.027535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764535+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:43:34.764535+0000","last_peered":"2023-08-25T04:43:34.764535+0000","last_clean":"2023-08-25T04:43:34.764535+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:43:34.764535+0000","last_undegraded":"2023-08-25T04:43:34.764535+0000","last_fullsized":"2023-08-25T04:43:34.764535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:00.744928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922718+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:43:34.922718+0000","last_peered":"2023-08-25T04:43:34.922718+0000","last_clean":"2023-08-25T04:43:34.922718+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:43:34.922718+0000","last_undegraded":"2023-08-25T04:43:34.922718+0000","last_fullsized":"2023-08-25T04:43:34.922718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:47:46.522466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765008+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:43:34.765008+0000","last_peered":"2023-08-25T04:43:34.765008+0000","last_clean":"2023-08-25T04:43:34.765008+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:43:34.765008+0000","last_undegraded":"2023-08-25T04:43:34.765008+0000","last_fullsized":"2023-08-25T04:43:34.765008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:08:38.855500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099337+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:43:35.099337+0000","last_peered":"2023-08-25T04:43:35.099337+0000","last_clean":"2023-08-25T04:43:35.099337+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:43:35.099337+0000","last_undegraded":"2023-08-25T04:43:35.099337+0000","last_fullsized":"2023-08-25T04:43:35.099337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:36.964244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765117+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:43:34.765117+0000","last_peered":"2023-08-25T04:43:34.765117+0000","last_clean":"2023-08-25T04:43:34.765117+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:43:34.765117+0000","last_undegraded":"2023-08-25T04:43:34.765117+0000","last_fullsized":"2023-08-25T04:43:34.765117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:00.051214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923189+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:43:34.923189+0000","last_peered":"2023-08-25T04:43:34.923189+0000","last_clean":"2023-08-25T04:43:34.923189+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:43:34.923189+0000","last_undegraded":"2023-08-25T04:43:34.923189+0000","last_fullsized":"2023-08-25T04:43:34.923189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:24.087589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762199+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:43:34.762199+0000","last_peered":"2023-08-25T04:43:34.762199+0000","last_clean":"2023-08-25T04:43:34.762199+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:43:34.762199+0000","last_undegraded":"2023-08-25T04:43:34.762199+0000","last_fullsized":"2023-08-25T04:43:34.762199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:23.029577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099138+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:43:35.099138+0000","last_peered":"2023-08-25T04:43:35.099138+0000","last_clean":"2023-08-25T04:43:35.099138+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:43:35.099138+0000","last_undegraded":"2023-08-25T04:43:35.099138+0000","last_fullsized":"2023-08-25T04:43:35.099138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:53.794589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764677+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:43:34.764677+0000","last_peered":"2023-08-25T04:43:34.764677+0000","last_clean":"2023-08-25T04:43:34.764677+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:43:34.764677+0000","last_undegraded":"2023-08-25T04:43:34.764677+0000","last_fullsized":"2023-08-25T04:43:34.764677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:07.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764876+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:43:34.764876+0000","last_peered":"2023-08-25T04:43:34.764876+0000","last_clean":"2023-08-25T04:43:34.764876+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:43:34.764876+0000","last_undegraded":"2023-08-25T04:43:34.764876+0000","last_fullsized":"2023-08-25T04:43:34.764876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:15:49.317438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098418+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:43:35.098418+0000","last_peered":"2023-08-25T04:43:35.098418+0000","last_clean":"2023-08-25T04:43:35.098418+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:43:35.098418+0000","last_undegraded":"2023-08-25T04:43:35.098418+0000","last_fullsized":"2023-08-25T04:43:35.098418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:53.277068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765316+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:43:34.765316+0000","last_peered":"2023-08-25T04:43:34.765316+0000","last_clean":"2023-08-25T04:43:34.765316+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:43:34.765316+0000","last_undegraded":"2023-08-25T04:43:34.765316+0000","last_fullsized":"2023-08-25T04:43:34.765316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:04.508809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761810+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:43:34.761810+0000","last_peered":"2023-08-25T04:43:34.761810+0000","last_clean":"2023-08-25T04:43:34.761810+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:43:34.761810+0000","last_undegraded":"2023-08-25T04:43:34.761810+0000","last_fullsized":"2023-08-25T04:43:34.761810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:10:02.382799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770010+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:43:34.770010+0000","last_peered":"2023-08-25T04:43:34.770010+0000","last_clean":"2023-08-25T04:43:34.770010+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:43:34.770010+0000","last_undegraded":"2023-08-25T04:43:34.770010+0000","last_fullsized":"2023-08-25T04:43:34.770010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:00:34.306505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769760+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:43:34.769760+0000","last_peered":"2023-08-25T04:43:34.769760+0000","last_clean":"2023-08-25T04:43:34.769760+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:43:34.769760+0000","last_undegraded":"2023-08-25T04:43:34.769760+0000","last_fullsized":"2023-08-25T04:43:34.769760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:23.911843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761620+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:43:34.761620+0000","last_peered":"2023-08-25T04:43:34.761620+0000","last_clean":"2023-08-25T04:43:34.761620+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:43:34.761620+0000","last_undegraded":"2023-08-25T04:43:34.761620+0000","last_fullsized":"2023-08-25T04:43:34.761620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:30.639568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765547+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:43:34.765547+0000","last_peered":"2023-08-25T04:43:34.765547+0000","last_clean":"2023-08-25T04:43:34.765547+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:43:34.765547+0000","last_undegraded":"2023-08-25T04:43:34.765547+0000","last_fullsized":"2023-08-25T04:43:34.765547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:24.659622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100513+0000","last_change":"2023-08-25T04:43:35.100513+0000","last_active":"2023-08-25T04:43:35.100513+0000","last_peered":"2023-08-25T04:43:35.100513+0000","last_clean":"2023-08-25T04:43:35.100513+0000","last_became_active":"2023-08-25T04:43:35.100067+0000","last_became_peered":"2023-08-25T04:43:35.100067+0000","last_unstale":"2023-08-25T04:43:35.100513+0000","last_undegraded":"2023-08-25T04:43:35.100513+0000","last_fullsized":"2023-08-25T04:43:35.100513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:23:03.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765693+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:43:34.765693+0000","last_peered":"2023-08-25T04:43:34.765693+0000","last_clean":"2023-08-25T04:43:34.765693+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:43:34.765693+0000","last_undegraded":"2023-08-25T04:43:34.765693+0000","last_fullsized":"2023-08-25T04:43:34.765693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:38:39.362078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765706+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:43:34.765706+0000","last_peered":"2023-08-25T04:43:34.765706+0000","last_clean":"2023-08-25T04:43:34.765706+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:43:34.765706+0000","last_undegraded":"2023-08-25T04:43:34.765706+0000","last_fullsized":"2023-08-25T04:43:34.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:25.041276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761550+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:43:34.761550+0000","last_peered":"2023-08-25T04:43:34.761550+0000","last_clean":"2023-08-25T04:43:34.761550+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:43:34.761550+0000","last_undegraded":"2023-08-25T04:43:34.761550+0000","last_fullsized":"2023-08-25T04:43:34.761550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:25.771337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765423+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:43:34.765423+0000","last_peered":"2023-08-25T04:43:34.765423+0000","last_clean":"2023-08-25T04:43:34.765423+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:43:34.765423+0000","last_undegraded":"2023-08-25T04:43:34.765423+0000","last_fullsized":"2023-08-25T04:43:34.765423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:27.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098820+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:43:35.098820+0000","last_peered":"2023-08-25T04:43:35.098820+0000","last_clean":"2023-08-25T04:43:35.098820+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:43:35.098820+0000","last_undegraded":"2023-08-25T04:43:35.098820+0000","last_fullsized":"2023-08-25T04:43:35.098820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:20.999420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761408+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:43:34.761408+0000","last_peered":"2023-08-25T04:43:34.761408+0000","last_clean":"2023-08-25T04:43:34.761408+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:43:34.761408+0000","last_undegraded":"2023-08-25T04:43:34.761408+0000","last_fullsized":"2023-08-25T04:43:34.761408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:14.562518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764854+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:43:34.764854+0000","last_peered":"2023-08-25T04:43:34.764854+0000","last_clean":"2023-08-25T04:43:34.764854+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:43:34.764854+0000","last_undegraded":"2023-08-25T04:43:34.764854+0000","last_fullsized":"2023-08-25T04:43:34.764854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:27.603878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099267+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:43:35.099267+0000","last_peered":"2023-08-25T04:43:35.099267+0000","last_clean":"2023-08-25T04:43:35.099267+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:43:35.099267+0000","last_undegraded":"2023-08-25T04:43:35.099267+0000","last_fullsized":"2023-08-25T04:43:35.099267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:28.457166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769245+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:43:34.769245+0000","last_peered":"2023-08-25T04:43:34.769245+0000","last_clean":"2023-08-25T04:43:34.769245+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:43:34.769245+0000","last_undegraded":"2023-08-25T04:43:34.769245+0000","last_fullsized":"2023-08-25T04:43:34.769245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:47.099556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":127,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.695688+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:44:46.695688+0000","last_peered":"2023-08-25T04:44:46.695688+0000","last_clean":"2023-08-25T04:44:46.695688+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:44:46.695688+0000","last_undegraded":"2023-08-25T04:44:46.695688+0000","last_fullsized":"2023-08-25T04:44:46.695688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:02.879296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765116+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:43:34.765116+0000","last_peered":"2023-08-25T04:43:34.765116+0000","last_clean":"2023-08-25T04:43:34.765116+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:43:34.765116+0000","last_undegraded":"2023-08-25T04:43:34.765116+0000","last_fullsized":"2023-08-25T04:43:34.765116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:18.764324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769440+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:43:34.769440+0000","last_peered":"2023-08-25T04:43:34.769440+0000","last_clean":"2023-08-25T04:43:34.769440+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:43:34.769440+0000","last_undegraded":"2023-08-25T04:43:34.769440+0000","last_fullsized":"2023-08-25T04:43:34.769440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:38.395114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764497+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:43:34.764497+0000","last_peered":"2023-08-25T04:43:34.764497+0000","last_clean":"2023-08-25T04:43:34.764497+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:43:34.764497+0000","last_undegraded":"2023-08-25T04:43:34.764497+0000","last_fullsized":"2023-08-25T04:43:34.764497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:29:44.193254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099118+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:43:35.099118+0000","last_peered":"2023-08-25T04:43:35.099118+0000","last_clean":"2023-08-25T04:43:35.099118+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:43:35.099118+0000","last_undegraded":"2023-08-25T04:43:35.099118+0000","last_fullsized":"2023-08-25T04:43:35.099118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.013041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765157+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:43:34.765157+0000","last_peered":"2023-08-25T04:43:34.765157+0000","last_clean":"2023-08-25T04:43:34.765157+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:43:34.765157+0000","last_undegraded":"2023-08-25T04:43:34.765157+0000","last_fullsized":"2023-08-25T04:43:34.765157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:14.536121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098445+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:43:35.098445+0000","last_peered":"2023-08-25T04:43:35.098445+0000","last_clean":"2023-08-25T04:43:35.098445+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:43:35.098445+0000","last_undegraded":"2023-08-25T04:43:35.098445+0000","last_fullsized":"2023-08-25T04:43:35.098445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:45.583226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764897+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:43:34.764897+0000","last_peered":"2023-08-25T04:43:34.764897+0000","last_clean":"2023-08-25T04:43:34.764897+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:43:34.764897+0000","last_undegraded":"2023-08-25T04:43:34.764897+0000","last_fullsized":"2023-08-25T04:43:34.764897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:04.925252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764614+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:43:34.764614+0000","last_peered":"2023-08-25T04:43:34.764614+0000","last_clean":"2023-08-25T04:43:34.764614+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:43:34.764614+0000","last_undegraded":"2023-08-25T04:43:34.764614+0000","last_fullsized":"2023-08-25T04:43:34.764614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:24.431309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098697+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:43:35.098697+0000","last_peered":"2023-08-25T04:43:35.098697+0000","last_clean":"2023-08-25T04:43:35.098697+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:43:35.098697+0000","last_undegraded":"2023-08-25T04:43:35.098697+0000","last_fullsized":"2023-08-25T04:43:35.098697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:26.668766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761101+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:43:34.761101+0000","last_peered":"2023-08-25T04:43:34.761101+0000","last_clean":"2023-08-25T04:43:34.761101+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:43:34.761101+0000","last_undegraded":"2023-08-25T04:43:34.761101+0000","last_fullsized":"2023-08-25T04:43:34.761101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:51.398185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765545+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:43:34.765545+0000","last_peered":"2023-08-25T04:43:34.765545+0000","last_clean":"2023-08-25T04:43:34.765545+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:43:34.765545+0000","last_undegraded":"2023-08-25T04:43:34.765545+0000","last_fullsized":"2023-08-25T04:43:34.765545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:28.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:47.696571+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:44:47.696571+0000","last_peered":"2023-08-25T04:44:47.696571+0000","last_clean":"2023-08-25T04:44:47.696571+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:44:47.696571+0000","last_undegraded":"2023-08-25T04:44:47.696571+0000","last_fullsized":"2023-08-25T04:44:47.696571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:44.974722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099154+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:43:35.099154+0000","last_peered":"2023-08-25T04:43:35.099154+0000","last_clean":"2023-08-25T04:43:35.099154+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:43:35.099154+0000","last_undegraded":"2023-08-25T04:43:35.099154+0000","last_fullsized":"2023-08-25T04:43:35.099154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:07.872117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761225+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:43:34.761225+0000","last_peered":"2023-08-25T04:43:34.761225+0000","last_clean":"2023-08-25T04:43:34.761225+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:43:34.761225+0000","last_undegraded":"2023-08-25T04:43:34.761225+0000","last_fullsized":"2023-08-25T04:43:34.761225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:18.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923185+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:43:34.923185+0000","last_peered":"2023-08-25T04:43:34.923185+0000","last_clean":"2023-08-25T04:43:34.923185+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:43:34.923185+0000","last_undegraded":"2023-08-25T04:43:34.923185+0000","last_fullsized":"2023-08-25T04:43:34.923185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:14.764732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766754+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:43:34.766754+0000","last_peered":"2023-08-25T04:43:34.766754+0000","last_clean":"2023-08-25T04:43:34.766754+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:43:34.766754+0000","last_undegraded":"2023-08-25T04:43:34.766754+0000","last_fullsized":"2023-08-25T04:43:34.766754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:01.393459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100106+0000","last_change":"2023-08-25T04:43:35.100106+0000","last_active":"2023-08-25T04:43:35.100106+0000","last_peered":"2023-08-25T04:43:35.100106+0000","last_clean":"2023-08-25T04:43:35.100106+0000","last_became_active":"2023-08-25T04:43:35.099546+0000","last_became_peered":"2023-08-25T04:43:35.099546+0000","last_unstale":"2023-08-25T04:43:35.100106+0000","last_undegraded":"2023-08-25T04:43:35.100106+0000","last_fullsized":"2023-08-25T04:43:35.100106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:25.055771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099268+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:43:35.099268+0000","last_peered":"2023-08-25T04:43:35.099268+0000","last_clean":"2023-08-25T04:43:35.099268+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:43:35.099268+0000","last_undegraded":"2023-08-25T04:43:35.099268+0000","last_fullsized":"2023-08-25T04:43:35.099268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:54.567632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098918+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:43:35.098918+0000","last_peered":"2023-08-25T04:43:35.098918+0000","last_clean":"2023-08-25T04:43:35.098918+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:43:35.098918+0000","last_undegraded":"2023-08-25T04:43:35.098918+0000","last_fullsized":"2023-08-25T04:43:35.098918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:47.455399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761107+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:43:34.761107+0000","last_peered":"2023-08-25T04:43:34.761107+0000","last_clean":"2023-08-25T04:43:34.761107+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:43:34.761107+0000","last_undegraded":"2023-08-25T04:43:34.761107+0000","last_fullsized":"2023-08-25T04:43:34.761107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:27:18.610077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:48.697368+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:44:48.697368+0000","last_peered":"2023-08-25T04:44:48.697368+0000","last_clean":"2023-08-25T04:44:48.697368+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:44:48.697368+0000","last_undegraded":"2023-08-25T04:44:48.697368+0000","last_fullsized":"2023-08-25T04:44:48.697368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:00.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":188,"num_read_kb":98,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765074+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:43:34.765074+0000","last_peered":"2023-08-25T04:43:34.765074+0000","last_clean":"2023-08-25T04:43:34.765074+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:43:34.765074+0000","last_undegraded":"2023-08-25T04:43:34.765074+0000","last_fullsized":"2023-08-25T04:43:34.765074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:07.231855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099639+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:43:35.099639+0000","last_peered":"2023-08-25T04:43:35.099639+0000","last_clean":"2023-08-25T04:43:35.099639+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:43:35.099639+0000","last_undegraded":"2023-08-25T04:43:35.099639+0000","last_fullsized":"2023-08-25T04:43:35.099639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:46:34.120949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769705+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:43:34.769705+0000","last_peered":"2023-08-25T04:43:34.769705+0000","last_clean":"2023-08-25T04:43:34.769705+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:43:34.769705+0000","last_undegraded":"2023-08-25T04:43:34.769705+0000","last_fullsized":"2023-08-25T04:43:34.769705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:00.803708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098704+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:43:35.098704+0000","last_peered":"2023-08-25T04:43:35.098704+0000","last_clean":"2023-08-25T04:43:35.098704+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:43:35.098704+0000","last_undegraded":"2023-08-25T04:43:35.098704+0000","last_fullsized":"2023-08-25T04:43:35.098704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:19:57.215688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.767465+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:43:34.767465+0000","last_peered":"2023-08-25T04:43:34.767465+0000","last_clean":"2023-08-25T04:43:34.767465+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:43:34.767465+0000","last_undegraded":"2023-08-25T04:43:34.767465+0000","last_fullsized":"2023-08-25T04:43:34.767465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:22.379000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098949+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:43:35.098949+0000","last_peered":"2023-08-25T04:43:35.098949+0000","last_clean":"2023-08-25T04:43:35.098949+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:43:35.098949+0000","last_undegraded":"2023-08-25T04:43:35.098949+0000","last_fullsized":"2023-08-25T04:43:35.098949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:38.602930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923061+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:43:34.923061+0000","last_peered":"2023-08-25T04:43:34.923061+0000","last_clean":"2023-08-25T04:43:34.923061+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:43:34.923061+0000","last_undegraded":"2023-08-25T04:43:34.923061+0000","last_fullsized":"2023-08-25T04:43:34.923061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:57.719232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761321+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:43:34.761321+0000","last_peered":"2023-08-25T04:43:34.761321+0000","last_clean":"2023-08-25T04:43:34.761321+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:43:34.761321+0000","last_undegraded":"2023-08-25T04:43:34.761321+0000","last_fullsized":"2023-08-25T04:43:34.761321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:45.122188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.718684+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:44:46.718684+0000","last_peered":"2023-08-25T04:44:46.718684+0000","last_clean":"2023-08-25T04:44:46.718684+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:44:46.718684+0000","last_undegraded":"2023-08-25T04:44:46.718684+0000","last_fullsized":"2023-08-25T04:44:46.718684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:03:39.958808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922765+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:43:34.922765+0000","last_peered":"2023-08-25T04:43:34.922765+0000","last_clean":"2023-08-25T04:43:34.922765+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:43:34.922765+0000","last_undegraded":"2023-08-25T04:43:34.922765+0000","last_fullsized":"2023-08-25T04:43:34.922765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:22.482680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099199+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:43:35.099199+0000","last_peered":"2023-08-25T04:43:35.099199+0000","last_clean":"2023-08-25T04:43:35.099199+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:43:35.099199+0000","last_undegraded":"2023-08-25T04:43:35.099199+0000","last_fullsized":"2023-08-25T04:43:35.099199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:46.918540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765654+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:43:34.765654+0000","last_peered":"2023-08-25T04:43:34.765654+0000","last_clean":"2023-08-25T04:43:34.765654+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:43:34.765654+0000","last_undegraded":"2023-08-25T04:43:34.765654+0000","last_fullsized":"2023-08-25T04:43:34.765654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:28.897000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:02.027948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098592+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:43:35.098592+0000","last_peered":"2023-08-25T04:43:35.098592+0000","last_clean":"2023-08-25T04:43:35.098592+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:43:35.098592+0000","last_undegraded":"2023-08-25T04:43:35.098592+0000","last_fullsized":"2023-08-25T04:43:35.098592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:19:33.883508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764534+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:43:34.764534+0000","last_peered":"2023-08-25T04:43:34.764534+0000","last_clean":"2023-08-25T04:43:34.764534+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:43:34.764534+0000","last_undegraded":"2023-08-25T04:43:34.764534+0000","last_fullsized":"2023-08-25T04:43:34.764534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:04:26.603958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766834+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:43:34.766834+0000","last_peered":"2023-08-25T04:43:34.766834+0000","last_clean":"2023-08-25T04:43:34.766834+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:43:34.766834+0000","last_undegraded":"2023-08-25T04:43:34.766834+0000","last_fullsized":"2023-08-25T04:43:34.766834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:35:18.420398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764942+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:43:34.764942+0000","last_peered":"2023-08-25T04:43:34.764942+0000","last_clean":"2023-08-25T04:43:34.764942+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:43:34.764942+0000","last_undegraded":"2023-08-25T04:43:34.764942+0000","last_fullsized":"2023-08-25T04:43:34.764942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:31.579981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761767+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:43:34.761767+0000","last_peered":"2023-08-25T04:43:34.761767+0000","last_clean":"2023-08-25T04:43:34.761767+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:43:34.761767+0000","last_undegraded":"2023-08-25T04:43:34.761767+0000","last_fullsized":"2023-08-25T04:43:34.761767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:32.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764777+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:43:34.764777+0000","last_peered":"2023-08-25T04:43:34.764777+0000","last_clean":"2023-08-25T04:43:34.764777+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:43:34.764777+0000","last_undegraded":"2023-08-25T04:43:34.764777+0000","last_fullsized":"2023-08-25T04:43:34.764777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:25:29.024128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099012+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:43:35.099012+0000","last_peered":"2023-08-25T04:43:35.099012+0000","last_clean":"2023-08-25T04:43:35.099012+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:43:35.099012+0000","last_undegraded":"2023-08-25T04:43:35.099012+0000","last_fullsized":"2023-08-25T04:43:35.099012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:17.119675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922819+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:43:34.922819+0000","last_peered":"2023-08-25T04:43:34.922819+0000","last_clean":"2023-08-25T04:43:34.922819+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:43:34.922819+0000","last_undegraded":"2023-08-25T04:43:34.922819+0000","last_fullsized":"2023-08-25T04:43:34.922819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:42.375965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769924+0000","last_change":"2023-08-25T04:43:34.769924+0000","last_active":"2023-08-25T04:43:34.769924+0000","last_peered":"2023-08-25T04:43:34.769924+0000","last_clean":"2023-08-25T04:43:34.769924+0000","last_became_active":"2023-08-25T04:43:34.769575+0000","last_became_peered":"2023-08-25T04:43:34.769575+0000","last_unstale":"2023-08-25T04:43:34.769924+0000","last_undegraded":"2023-08-25T04:43:34.769924+0000","last_fullsized":"2023-08-25T04:43:34.769924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:32.013614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099091+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:43:35.099091+0000","last_peered":"2023-08-25T04:43:35.099091+0000","last_clean":"2023-08-25T04:43:35.099091+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:43:35.099091+0000","last_undegraded":"2023-08-25T04:43:35.099091+0000","last_fullsized":"2023-08-25T04:43:35.099091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:32:02.438964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.772835+0000","last_change":"2023-08-25T04:43:34.771233+0000","last_active":"2023-08-25T04:43:34.772835+0000","last_peered":"2023-08-25T04:43:34.772835+0000","last_clean":"2023-08-25T04:43:34.772835+0000","last_became_active":"2023-08-25T04:43:34.770887+0000","last_became_peered":"2023-08-25T04:43:34.770887+0000","last_unstale":"2023-08-25T04:43:34.772835+0000","last_undegraded":"2023-08-25T04:43:34.772835+0000","last_fullsized":"2023-08-25T04:43:34.772835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:08.119078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":122,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.717700+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:44:46.717700+0000","last_peered":"2023-08-25T04:44:46.717700+0000","last_clean":"2023-08-25T04:44:46.717700+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:44:46.717700+0000","last_undegraded":"2023-08-25T04:44:46.717700+0000","last_fullsized":"2023-08-25T04:44:46.717700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:43.271628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922975+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:43:34.922975+0000","last_peered":"2023-08-25T04:43:34.922975+0000","last_clean":"2023-08-25T04:43:34.922975+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:43:34.922975+0000","last_undegraded":"2023-08-25T04:43:34.922975+0000","last_fullsized":"2023-08-25T04:43:34.922975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:04.252263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764688+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:43:34.764688+0000","last_peered":"2023-08-25T04:43:34.764688+0000","last_clean":"2023-08-25T04:43:34.764688+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:43:34.764688+0000","last_undegraded":"2023-08-25T04:43:34.764688+0000","last_fullsized":"2023-08-25T04:43:34.764688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:33:20.016878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761743+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:43:34.761743+0000","last_peered":"2023-08-25T04:43:34.761743+0000","last_clean":"2023-08-25T04:43:34.761743+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:43:34.761743+0000","last_undegraded":"2023-08-25T04:43:34.761743+0000","last_fullsized":"2023-08-25T04:43:34.761743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:10.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922646+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:43:34.922646+0000","last_peered":"2023-08-25T04:43:34.922646+0000","last_clean":"2023-08-25T04:43:34.922646+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:43:34.922646+0000","last_undegraded":"2023-08-25T04:43:34.922646+0000","last_fullsized":"2023-08-25T04:43:34.922646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:20.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099295+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:43:35.099295+0000","last_peered":"2023-08-25T04:43:35.099295+0000","last_clean":"2023-08-25T04:43:35.099295+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:43:35.099295+0000","last_undegraded":"2023-08-25T04:43:35.099295+0000","last_fullsized":"2023-08-25T04:43:35.099295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.332881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098261+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:43:35.098261+0000","last_peered":"2023-08-25T04:43:35.098261+0000","last_clean":"2023-08-25T04:43:35.098261+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:43:35.098261+0000","last_undegraded":"2023-08-25T04:43:35.098261+0000","last_fullsized":"2023-08-25T04:43:35.098261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:51:31.902430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922785+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:43:34.922785+0000","last_peered":"2023-08-25T04:43:34.922785+0000","last_clean":"2023-08-25T04:43:34.922785+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:43:34.922785+0000","last_undegraded":"2023-08-25T04:43:34.922785+0000","last_fullsized":"2023-08-25T04:43:34.922785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:47.704937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765952+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:43:34.765952+0000","last_peered":"2023-08-25T04:43:34.765952+0000","last_clean":"2023-08-25T04:43:34.765952+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:43:34.765952+0000","last_undegraded":"2023-08-25T04:43:34.765952+0000","last_fullsized":"2023-08-25T04:43:34.765952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:38:36.745026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765218+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:43:34.765218+0000","last_peered":"2023-08-25T04:43:34.765218+0000","last_clean":"2023-08-25T04:43:34.765218+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:43:34.765218+0000","last_undegraded":"2023-08-25T04:43:34.765218+0000","last_fullsized":"2023-08-25T04:43:34.765218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:15:24.598228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:14:26.761777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764760+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:43:34.764760+0000","last_peered":"2023-08-25T04:43:34.764760+0000","last_clean":"2023-08-25T04:43:34.764760+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:43:34.764760+0000","last_undegraded":"2023-08-25T04:43:34.764760+0000","last_fullsized":"2023-08-25T04:43:34.764760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:23.436529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.777294+0000","last_change":"2023-08-25T04:43:34.777294+0000","last_active":"2023-08-25T04:43:34.777294+0000","last_peered":"2023-08-25T04:43:34.777294+0000","last_clean":"2023-08-25T04:43:34.777294+0000","last_became_active":"2023-08-25T04:43:34.768036+0000","last_became_peered":"2023-08-25T04:43:34.768036+0000","last_unstale":"2023-08-25T04:43:34.777294+0000","last_undegraded":"2023-08-25T04:43:34.777294+0000","last_fullsized":"2023-08-25T04:43:34.777294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:52.521174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765705+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:43:34.765705+0000","last_peered":"2023-08-25T04:43:34.765705+0000","last_clean":"2023-08-25T04:43:34.765705+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:43:34.765705+0000","last_undegraded":"2023-08-25T04:43:34.765705+0000","last_fullsized":"2023-08-25T04:43:34.765705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:34.577696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768990+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:43:34.768990+0000","last_peered":"2023-08-25T04:43:34.768990+0000","last_clean":"2023-08-25T04:43:34.768990+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:43:34.768990+0000","last_undegraded":"2023-08-25T04:43:34.768990+0000","last_fullsized":"2023-08-25T04:43:34.768990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:09.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761921+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:43:34.761921+0000","last_peered":"2023-08-25T04:43:34.761921+0000","last_clean":"2023-08-25T04:43:34.761921+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:43:34.761921+0000","last_undegraded":"2023-08-25T04:43:34.761921+0000","last_fullsized":"2023-08-25T04:43:34.761921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:16.228950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764889+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:43:34.764889+0000","last_peered":"2023-08-25T04:43:34.764889+0000","last_clean":"2023-08-25T04:43:34.764889+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:43:34.764889+0000","last_undegraded":"2023-08-25T04:43:34.764889+0000","last_fullsized":"2023-08-25T04:43:34.764889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:14.043346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768669+0000","last_change":"2023-08-25T04:43:34.768669+0000","last_active":"2023-08-25T04:43:34.768669+0000","last_peered":"2023-08-25T04:43:34.768669+0000","last_clean":"2023-08-25T04:43:34.768669+0000","last_became_active":"2023-08-25T04:43:34.768275+0000","last_became_peered":"2023-08-25T04:43:34.768275+0000","last_unstale":"2023-08-25T04:43:34.768669+0000","last_undegraded":"2023-08-25T04:43:34.768669+0000","last_fullsized":"2023-08-25T04:43:34.768669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:39:53.898292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,11],"acting":[4,8,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098506+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:43:35.098506+0000","last_peered":"2023-08-25T04:43:35.098506+0000","last_clean":"2023-08-25T04:43:35.098506+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:43:35.098506+0000","last_undegraded":"2023-08-25T04:43:35.098506+0000","last_fullsized":"2023-08-25T04:43:35.098506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:19.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764296+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:43:34.764296+0000","last_peered":"2023-08-25T04:43:34.764296+0000","last_clean":"2023-08-25T04:43:34.764296+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:43:34.764296+0000","last_undegraded":"2023-08-25T04:43:34.764296+0000","last_fullsized":"2023-08-25T04:43:34.764296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:11:04.726743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761549+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:43:34.761549+0000","last_peered":"2023-08-25T04:43:34.761549+0000","last_clean":"2023-08-25T04:43:34.761549+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:43:34.761549+0000","last_undegraded":"2023-08-25T04:43:34.761549+0000","last_fullsized":"2023-08-25T04:43:34.761549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:48.355591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765742+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:43:34.765742+0000","last_peered":"2023-08-25T04:43:34.765742+0000","last_clean":"2023-08-25T04:43:34.765742+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:43:34.765742+0000","last_undegraded":"2023-08-25T04:43:34.765742+0000","last_fullsized":"2023-08-25T04:43:34.765742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:04.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766384+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:43:34.766384+0000","last_peered":"2023-08-25T04:43:34.766384+0000","last_clean":"2023-08-25T04:43:34.766384+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:43:34.766384+0000","last_undegraded":"2023-08-25T04:43:34.766384+0000","last_fullsized":"2023-08-25T04:43:34.766384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:02:40.882696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761618+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:43:34.761618+0000","last_peered":"2023-08-25T04:43:34.761618+0000","last_clean":"2023-08-25T04:43:34.761618+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:43:34.761618+0000","last_undegraded":"2023-08-25T04:43:34.761618+0000","last_fullsized":"2023-08-25T04:43:34.761618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:23.594133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.779656+0000","last_change":"2023-08-25T04:43:34.779656+0000","last_active":"2023-08-25T04:43:34.779656+0000","last_peered":"2023-08-25T04:43:34.779656+0000","last_clean":"2023-08-25T04:43:34.779656+0000","last_became_active":"2023-08-25T04:43:34.770821+0000","last_became_peered":"2023-08-25T04:43:34.770821+0000","last_unstale":"2023-08-25T04:43:34.779656+0000","last_undegraded":"2023-08-25T04:43:34.779656+0000","last_fullsized":"2023-08-25T04:43:34.779656+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:08:31.402768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766129+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:43:34.766129+0000","last_peered":"2023-08-25T04:43:34.766129+0000","last_clean":"2023-08-25T04:43:34.766129+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:43:34.766129+0000","last_undegraded":"2023-08-25T04:43:34.766129+0000","last_fullsized":"2023-08-25T04:43:34.766129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:18.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761763+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:43:34.761763+0000","last_peered":"2023-08-25T04:43:34.761763+0000","last_clean":"2023-08-25T04:43:34.761763+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:43:34.761763+0000","last_undegraded":"2023-08-25T04:43:34.761763+0000","last_fullsized":"2023-08-25T04:43:34.761763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:14.914419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765951+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:43:34.765951+0000","last_peered":"2023-08-25T04:43:34.765951+0000","last_clean":"2023-08-25T04:43:34.765951+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:43:34.765951+0000","last_undegraded":"2023-08-25T04:43:34.765951+0000","last_fullsized":"2023-08-25T04:43:34.765951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:07.289831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922875+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:43:34.922875+0000","last_peered":"2023-08-25T04:43:34.922875+0000","last_clean":"2023-08-25T04:43:34.922875+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:43:34.922875+0000","last_undegraded":"2023-08-25T04:43:34.922875+0000","last_fullsized":"2023-08-25T04:43:34.922875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:13.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098664+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:43:35.098664+0000","last_peered":"2023-08-25T04:43:35.098664+0000","last_clean":"2023-08-25T04:43:35.098664+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:43:35.098664+0000","last_undegraded":"2023-08-25T04:43:35.098664+0000","last_fullsized":"2023-08-25T04:43:35.098664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:02:41.742505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769588+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:43:34.769588+0000","last_peered":"2023-08-25T04:43:34.769588+0000","last_clean":"2023-08-25T04:43:34.769588+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:43:34.769588+0000","last_undegraded":"2023-08-25T04:43:34.769588+0000","last_fullsized":"2023-08-25T04:43:34.769588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:56.254087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923325+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:43:34.923325+0000","last_peered":"2023-08-25T04:43:34.923325+0000","last_clean":"2023-08-25T04:43:34.923325+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:43:34.923325+0000","last_undegraded":"2023-08-25T04:43:34.923325+0000","last_fullsized":"2023-08-25T04:43:34.923325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:27.560915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764860+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:43:34.764860+0000","last_peered":"2023-08-25T04:43:34.764860+0000","last_clean":"2023-08-25T04:43:34.764860+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:43:34.764860+0000","last_undegraded":"2023-08-25T04:43:34.764860+0000","last_fullsized":"2023-08-25T04:43:34.764860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:05.061144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099452+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:43:35.099452+0000","last_peered":"2023-08-25T04:43:35.099452+0000","last_clean":"2023-08-25T04:43:35.099452+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:43:35.099452+0000","last_undegraded":"2023-08-25T04:43:35.099452+0000","last_fullsized":"2023-08-25T04:43:35.099452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:28.319336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923060+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:43:34.923060+0000","last_peered":"2023-08-25T04:43:34.923060+0000","last_clean":"2023-08-25T04:43:34.923060+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:43:34.923060+0000","last_undegraded":"2023-08-25T04:43:34.923060+0000","last_fullsized":"2023-08-25T04:43:34.923060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:25.511917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098448+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:43:35.098448+0000","last_peered":"2023-08-25T04:43:35.098448+0000","last_clean":"2023-08-25T04:43:35.098448+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:43:35.098448+0000","last_undegraded":"2023-08-25T04:43:35.098448+0000","last_fullsized":"2023-08-25T04:43:35.098448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:15.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099187+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:43:35.099187+0000","last_peered":"2023-08-25T04:43:35.099187+0000","last_clean":"2023-08-25T04:43:35.099187+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:43:35.099187+0000","last_undegraded":"2023-08-25T04:43:35.099187+0000","last_fullsized":"2023-08-25T04:43:35.099187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:00.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923329+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:43:34.923329+0000","last_peered":"2023-08-25T04:43:34.923329+0000","last_clean":"2023-08-25T04:43:34.923329+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:43:34.923329+0000","last_undegraded":"2023-08-25T04:43:34.923329+0000","last_fullsized":"2023-08-25T04:43:34.923329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:03.933897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766391+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:43:34.766391+0000","last_peered":"2023-08-25T04:43:34.766391+0000","last_clean":"2023-08-25T04:43:34.766391+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:43:34.766391+0000","last_undegraded":"2023-08-25T04:43:34.766391+0000","last_fullsized":"2023-08-25T04:43:34.766391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:42.835852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769164+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:43:34.769164+0000","last_peered":"2023-08-25T04:43:34.769164+0000","last_clean":"2023-08-25T04:43:34.769164+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:43:34.769164+0000","last_undegraded":"2023-08-25T04:43:34.769164+0000","last_fullsized":"2023-08-25T04:43:34.769164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:38.408069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769152+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:43:34.769152+0000","last_peered":"2023-08-25T04:43:34.769152+0000","last_clean":"2023-08-25T04:43:34.769152+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:43:34.769152+0000","last_undegraded":"2023-08-25T04:43:34.769152+0000","last_fullsized":"2023-08-25T04:43:34.769152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:25.658427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100100+0000","last_change":"2023-08-25T04:43:35.100100+0000","last_active":"2023-08-25T04:43:35.100100+0000","last_peered":"2023-08-25T04:43:35.100100+0000","last_clean":"2023-08-25T04:43:35.100100+0000","last_became_active":"2023-08-25T04:43:35.099734+0000","last_became_peered":"2023-08-25T04:43:35.099734+0000","last_unstale":"2023-08-25T04:43:35.100100+0000","last_undegraded":"2023-08-25T04:43:35.100100+0000","last_fullsized":"2023-08-25T04:43:35.100100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:17.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099291+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:43:35.099291+0000","last_peered":"2023-08-25T04:43:35.099291+0000","last_clean":"2023-08-25T04:43:35.099291+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:43:35.099291+0000","last_undegraded":"2023-08-25T04:43:35.099291+0000","last_fullsized":"2023-08-25T04:43:35.099291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:05:39.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766561+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:43:34.766561+0000","last_peered":"2023-08-25T04:43:34.766561+0000","last_clean":"2023-08-25T04:43:34.766561+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:43:34.766561+0000","last_undegraded":"2023-08-25T04:43:34.766561+0000","last_fullsized":"2023-08-25T04:43:34.766561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:47:14.892189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770202+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:43:34.770202+0000","last_peered":"2023-08-25T04:43:34.770202+0000","last_clean":"2023-08-25T04:43:34.770202+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:43:34.770202+0000","last_undegraded":"2023-08-25T04:43:34.770202+0000","last_fullsized":"2023-08-25T04:43:34.770202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:31.428150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765901+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:43:34.765901+0000","last_peered":"2023-08-25T04:43:34.765901+0000","last_clean":"2023-08-25T04:43:34.765901+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:43:34.765901+0000","last_undegraded":"2023-08-25T04:43:34.765901+0000","last_fullsized":"2023-08-25T04:43:34.765901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:04.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:13.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765879+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:43:34.765879+0000","last_peered":"2023-08-25T04:43:34.765879+0000","last_clean":"2023-08-25T04:43:34.765879+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:43:34.765879+0000","last_undegraded":"2023-08-25T04:43:34.765879+0000","last_fullsized":"2023-08-25T04:43:34.765879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:03:35.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:15:06.086740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098646+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:43:35.098646+0000","last_peered":"2023-08-25T04:43:35.098646+0000","last_clean":"2023-08-25T04:43:35.098646+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:43:35.098646+0000","last_undegraded":"2023-08-25T04:43:35.098646+0000","last_fullsized":"2023-08-25T04:43:35.098646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:42.910594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.771119+0000","last_change":"2023-08-25T04:43:34.771119+0000","last_active":"2023-08-25T04:43:34.771119+0000","last_peered":"2023-08-25T04:43:34.771119+0000","last_clean":"2023-08-25T04:43:34.771119+0000","last_became_active":"2023-08-25T04:43:34.770717+0000","last_became_peered":"2023-08-25T04:43:34.770717+0000","last_unstale":"2023-08-25T04:43:34.771119+0000","last_undegraded":"2023-08-25T04:43:34.771119+0000","last_fullsized":"2023-08-25T04:43:34.771119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:42.596435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":576,"num_read_kb":302,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":483313,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":1.018,"15min":1.018},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90500000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70599999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71099999999999997,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.54700000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.496}]},{"osd":5,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84999999999999998,"5min":1.006,"15min":1.006},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.875,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.64800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":1.099,"15min":1.099},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.69999999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.66300000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79400000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.76300000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.85699999999999998,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.72099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97899999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72499999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1078519,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91100000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.036,"15min":1.036},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.91000000000000003,"5min":1.167,"15min":1.167},"last":0.80000000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77700000000000002},"min":{"1min":0.64500000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.72399999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841580,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483549,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.61299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.432},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.745},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84899999999999998,"5min":1.266,"15min":1.266},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70599999999999996,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.73799999999999999,"15min":0.78200000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69799999999999995,"15min":0.73299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.71899999999999997},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.628}]},{"osd":4,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80500000000000005,"15min":0.86499999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88200000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.85599999999999998},"min":{"1min":0.501,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.93799999999999994},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.87,"15min":0.92300000000000004},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.873,"15min":0.92700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.413,"15min":1.413},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.95999999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.47,"15min":1.47},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.876}]},{"osd":11,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103217,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483138,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60299999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.441,"15min":0.41999999999999998},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.746,"15min":0.746},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8730000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.871}]},{"osd":4,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.754}]},{"osd":5,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83199999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93200000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70899999999999996,"15min":0.68400000000000005},"min":{"1min":0.57899999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Aug 25 04:44:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.82599999999999996,"15min":0.78100000000000003},"min":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.9119999999999999,"5min":4.9119999999999999,"15min":4.9119999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.83599999999999997,"15min":0.78800000000000003},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.9569999999999999,"5min":3.9569999999999999,"15min":3.9569999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.61399999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.75},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97399999999999998,"5min":1.024,"15min":1.024},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86199999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074818,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.41099999999999998},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.876,"15min":0.876},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.628},"min":{"1min":0.51900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71799999999999997,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95499999999999996,"5min":0.996,"15min":0.996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.623}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.8980000000000001,"5min":3.8980000000000001,"15min":3.8980000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":0.79800000000000004},"min":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.014,"15min":1.014},"last":0.67400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.78200000000000003,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96599999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76200000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482821,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.57799999999999996,"15min":0.50900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.347,"5min":3.347,"15min":3.347},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.46899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68600000000000005,"15min":0.66100000000000003},"min":{"1min":0.53700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.745,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.69799999999999995,"15min":0.63100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.498},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.69299999999999995,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.625}]},{"osd":4,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.77700000000000002,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.502,"5min":0.442,"15min":0.442},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0329999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.80700000000000005,"15min":0.78500000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.84699999999999998,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.409,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.56299999999999994,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.1020000000000001,"5min":0.93000000000000005,"15min":0.90700000000000003},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.746}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91300000000000003,"15min":0.90600000000000003},"min":{"1min":0.68700000000000006,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.82699999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.873},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.83999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":486300,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96299999999999997,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.495}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.52600000000000002},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":1.121,"15min":1.121},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59199999999999997,"15min":0.51200000000000001},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":1.548,"15min":1.548},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.42499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.401,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.59899999999999998},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.39800000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.436},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72299999999999998,"5min":1.153,"15min":1.153},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96999999999999997,"5min":1.613,"15min":1.613},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69699999999999995,"15min":0.66400000000000003},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.2210000000000001,"5min":1.597,"15min":1.597},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.2130000000000001,"5min":1.26,"15min":1.26},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.75,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.375,"5min":11.375,"15min":11.375},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82899999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83999999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40799999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.76400000000000001,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.367,"5min":3.367,"15min":3.367},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.83499999999999996,"15min":0.79700000000000004},"min":{"1min":0.60599999999999998,"5min":0.437,"15min":0.437},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.79400000000000004,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93000000000000005,"5min":1.302,"15min":1.302},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92100000000000004,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.79500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":484209,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.67300000000000004,"15min":0.54700000000000004},"min":{"1min":0.48699999999999999,"5min":0.216,"15min":0.216},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.624,"15min":0.54600000000000004},"min":{"1min":0.51600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.53500000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95699999999999996,"5min":1.194,"15min":1.194},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.63,"15min":0.57199999999999995},"min":{"1min":0.52100000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.499}]},{"osd":4,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.69299999999999995,"15min":0.60399999999999998},"min":{"1min":0.52000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.5960000000000001,"5min":4.5960000000000001,"15min":4.5960000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.48899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.68400000000000005,"15min":0.63400000000000001},"min":{"1min":0.56999999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94799999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.028,"5min":1.365,"15min":1.365},"last":1.0029999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83399999999999996,"15min":0.82099999999999995},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.747},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.98799999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85799999999999998,"15min":0.82699999999999996},"min":{"1min":0.72099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93899999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.68799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.73399999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.087,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.96499999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":488181,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.53000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90500000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89500000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.622}]},{"osd":4,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.45100000000000001},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.45000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.61799999999999999},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96399999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63300000000000001,"15min":0.59199999999999997},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89900000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47899999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.622,"5min":0.44,"15min":0.44},"max":{"1min":0.98299999999999998,"5min":1.071,"15min":1.071},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.71299999999999997},"min":{"1min":0.59899999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.59999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80800000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.747},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83099999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483015,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.40100000000000002},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.42799999999999999},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.95099999999999996}]},{"osd":1,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.58299999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.341,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":1.53,"15min":1.53},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.573,"15min":1.573},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.628},"min":{"1min":0.52000000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.4410000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Aug 25 04:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.73999999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.077,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.747,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.337,"15min":1.337},"last":0.621}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.85899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.495,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.67800000000000005}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.88900000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.145},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1020000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66800000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.64600000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":483375,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41399999999999998,"15min":0.36199999999999999},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.753,"15min":0.753},"last":0.374},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.501},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.625,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84999999999999998,"5min":1.99,"15min":1.99},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.55700000000000005},"min":{"1min":0.53400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88200000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.67600000000000005},"min":{"1min":0.69399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84799999999999998},"min":{"1min":0.64000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0880000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.099,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.73599999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80200000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1074833,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.73299999999999998,"15min":0.59399999999999997},"min":{"1min":0.42099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":16.949999999999999,"5min":16.949999999999999,"15min":16.949999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.52600000000000002},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56499999999999995,"15min":0.50900000000000001},"min":{"1min":0.48299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.375},"min":{"1min":0.33200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70799999999999996,"5min":0.755,"15min":0.755},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.40000000000000002},"min":{"1min":0.31900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67300000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.56200000000000006},"min":{"1min":0.51700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.876,"5min":0.88,"15min":0.88},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.66600000000000004,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.57299999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.60799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.88200000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.071},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.79900000000000004,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":5.8479999999999999,"5min":5.8479999999999999,"15min":5.8479999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.61599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.72999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:44:50.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T04:44:51.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 -- 172.21.15.145:0/1849251550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 msgr2=0x7fcfd0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/1849251550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0101f70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009990 tx=0x7fcfbc009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 -- 172.21.15.145:0/1849251550 shutdown_connections 2023-08-25T04:44:51.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/1849251550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/1849251550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/1849251550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0101f70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 -- 172.21.15.145:0/1849251550 >> 172.21.15.145:0/1849251550 conn(0x7fcfd00fbe50 msgr2=0x7fcfd00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:51.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 -- 172.21.15.145:0/1849251550 shutdown_connections 2023-08-25T04:44:51.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.034+0000 7fcfd6dff700 1 -- 172.21.15.145:0/1849251550 wait complete. 2023-08-25T04:44:51.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.035+0000 7fcfd6dff700 1 Processor -- start 2023-08-25T04:44:51.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.035+0000 7fcfd6dff700 1 -- start start 2023-08-25T04:44:51.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd019e160 con 0x7fcfd01024b0 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd019e2a0 con 0x7fcfd0100920 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd6dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd019e430 con 0x7fcfd0101b10 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd4b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54658/0 (socket says 172.21.15.145:54658) 2023-08-25T04:44:51.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd4b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60904/0 (socket says 172.21.15.145:60904) 2023-08-25T04:44:51.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 -- 172.21.15.145:0/3605496392 learned_addr learned my addr 172.21.15.145:0/3605496392 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:51.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfcffff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 -- 172.21.15.145:0/3605496392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 msgr2=0x7fcfd0196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 -- 172.21.15.145:0/3605496392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 msgr2=0x7fcfd01965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd539c700 1 -- 172.21.15.145:0/3605496392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0095f0 con 0x7fcfd01024b0 2023-08-25T04:44:51.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.036+0000 7fcfd4b9b700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:51.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfd539c700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800ef30 tx=0x7fcfc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:51.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8018070 con 0x7fcfd01024b0 2023-08-25T04:44:51.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd0072b50 con 0x7fcfd01024b0 2023-08-25T04:44:51.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfc800f040 con 0x7fcfd01024b0 2023-08-25T04:44:51.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800cd00 con 0x7fcfd01024b0 2023-08-25T04:44:51.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.037+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0073060 con 0x7fcfd01024b0 2023-08-25T04:44:51.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.038+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0107eb0 con 0x7fcfd01024b0 2023-08-25T04:44:51.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.040+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fcfc8003a00 con 0x7fcfd01024b0 2023-08-25T04:44:51.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.042+0000 7fcfcdffb700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 0x7fcfc00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.042+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcfc80cb0f0 con 0x7fcfd01024b0 2023-08-25T04:44:51.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.042+0000 7fcfd4b9b700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 0x7fcfc00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.043+0000 7fcfd4b9b700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 0x7fcfc00a2460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0101970 tx=0x7fcfc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:51.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.046+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fcfc8094bb0 con 0x7fcfd01024b0 2023-08-25T04:44:51.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:50 smithi145 ceph-mon[97681]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:44:51.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:50 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1830292707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:51.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:50 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3945445202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:51.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:50 smithi162 ceph-mon[130941]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:44:51.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:50 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1830292707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:51.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:50 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3945445202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:51.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:50 smithi183 ceph-mon[131414]: pgmap v327: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:44:51.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:50 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1830292707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:51.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:50 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3945445202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:51.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.243+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfd002cd70 con 0x7fcfc009ffa0 2023-08-25T04:44:51.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.255+0000 7fcfcdffb700 1 -- 172.21.15.145:0/3605496392 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427232 (secure 0 0 0) 0x7fcfd002cd70 con 0x7fcfc009ffa0 2023-08-25T04:44:51.263 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:51.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 msgr2=0x7fcfc00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 0x7fcfc00a2460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0101970 tx=0x7fcfc4009260 comp rx=0 tx=0).stop 2023-08-25T04:44:51.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 msgr2=0x7fcfd019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800ef30 tx=0x7fcfc800c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 shutdown_connections 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fcfc009ffa0 0x7fcfc00a2460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfd01024b0 0x7fcfd019db90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfd0100920 0x7fcfd01965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 --2- 172.21.15.145:0/3605496392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0101b10 0x7fcfd0196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 >> 172.21.15.145:0/3605496392 conn(0x7fcfd00fbe50 msgr2=0x7fcfd0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:51.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 shutdown_connections 2023-08-25T04:44:51.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.261+0000 7fcfd6dff700 1 -- 172.21.15.145:0/3605496392 wait complete. 2023-08-25T04:44:51.268 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:44:51.282 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-08-25T04:44:49.714433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":712,"num_read_kb":421,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":11652,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":11931648,"data_stored":7586071,"data_compressed":318375,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002989"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765465+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:43:34.765465+0000","last_peered":"2023-08-25T04:43:34.765465+0000","last_clean":"2023-08-25T04:43:34.765465+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:43:34.765465+0000","last_undegraded":"2023-08-25T04:43:34.765465+0000","last_fullsized":"2023-08-25T04:43:34.765465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:59.835651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764768+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:43:34.764768+0000","last_peered":"2023-08-25T04:43:34.764768+0000","last_clean":"2023-08-25T04:43:34.764768+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:43:34.764768+0000","last_undegraded":"2023-08-25T04:43:34.764768+0000","last_fullsized":"2023-08-25T04:43:34.764768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:56:09.848052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766305+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:43:34.766305+0000","last_peered":"2023-08-25T04:43:34.766305+0000","last_clean":"2023-08-25T04:43:34.766305+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:43:34.766305+0000","last_undegraded":"2023-08-25T04:43:34.766305+0000","last_fullsized":"2023-08-25T04:43:34.766305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:56.683325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762343+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:43:34.762343+0000","last_peered":"2023-08-25T04:43:34.762343+0000","last_clean":"2023-08-25T04:43:34.762343+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:43:34.762343+0000","last_undegraded":"2023-08-25T04:43:34.762343+0000","last_fullsized":"2023-08-25T04:43:34.762343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:57:35.343381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766096+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:43:34.766096+0000","last_peered":"2023-08-25T04:43:34.766096+0000","last_clean":"2023-08-25T04:43:34.766096+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:43:34.766096+0000","last_undegraded":"2023-08-25T04:43:34.766096+0000","last_fullsized":"2023-08-25T04:43:34.766096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:53:26.027535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764535+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:43:34.764535+0000","last_peered":"2023-08-25T04:43:34.764535+0000","last_clean":"2023-08-25T04:43:34.764535+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:43:34.764535+0000","last_undegraded":"2023-08-25T04:43:34.764535+0000","last_fullsized":"2023-08-25T04:43:34.764535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:00.744928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922718+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:43:34.922718+0000","last_peered":"2023-08-25T04:43:34.922718+0000","last_clean":"2023-08-25T04:43:34.922718+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:43:34.922718+0000","last_undegraded":"2023-08-25T04:43:34.922718+0000","last_fullsized":"2023-08-25T04:43:34.922718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:47:46.522466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765008+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:43:34.765008+0000","last_peered":"2023-08-25T04:43:34.765008+0000","last_clean":"2023-08-25T04:43:34.765008+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:43:34.765008+0000","last_undegraded":"2023-08-25T04:43:34.765008+0000","last_fullsized":"2023-08-25T04:43:34.765008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:08:38.855500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099337+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:43:35.099337+0000","last_peered":"2023-08-25T04:43:35.099337+0000","last_clean":"2023-08-25T04:43:35.099337+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:43:35.099337+0000","last_undegraded":"2023-08-25T04:43:35.099337+0000","last_fullsized":"2023-08-25T04:43:35.099337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:36.964244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765117+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:43:34.765117+0000","last_peered":"2023-08-25T04:43:34.765117+0000","last_clean":"2023-08-25T04:43:34.765117+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:43:34.765117+0000","last_undegraded":"2023-08-25T04:43:34.765117+0000","last_fullsized":"2023-08-25T04:43:34.765117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:00.051214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923189+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:43:34.923189+0000","last_peered":"2023-08-25T04:43:34.923189+0000","last_clean":"2023-08-25T04:43:34.923189+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:43:34.923189+0000","last_undegraded":"2023-08-25T04:43:34.923189+0000","last_fullsized":"2023-08-25T04:43:34.923189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:24.087589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762199+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:43:34.762199+0000","last_peered":"2023-08-25T04:43:34.762199+0000","last_clean":"2023-08-25T04:43:34.762199+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:43:34.762199+0000","last_undegraded":"2023-08-25T04:43:34.762199+0000","last_fullsized":"2023-08-25T04:43:34.762199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:23.029577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099138+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:43:35.099138+0000","last_peered":"2023-08-25T04:43:35.099138+0000","last_clean":"2023-08-25T04:43:35.099138+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:43:35.099138+0000","last_undegraded":"2023-08-25T04:43:35.099138+0000","last_fullsized":"2023-08-25T04:43:35.099138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:53.794589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764677+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:43:34.764677+0000","last_peered":"2023-08-25T04:43:34.764677+0000","last_clean":"2023-08-25T04:43:34.764677+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:43:34.764677+0000","last_undegraded":"2023-08-25T04:43:34.764677+0000","last_fullsized":"2023-08-25T04:43:34.764677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:07.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764876+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:43:34.764876+0000","last_peered":"2023-08-25T04:43:34.764876+0000","last_clean":"2023-08-25T04:43:34.764876+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:43:34.764876+0000","last_undegraded":"2023-08-25T04:43:34.764876+0000","last_fullsized":"2023-08-25T04:43:34.764876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:15:49.317438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098418+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:43:35.098418+0000","last_peered":"2023-08-25T04:43:35.098418+0000","last_clean":"2023-08-25T04:43:35.098418+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:43:35.098418+0000","last_undegraded":"2023-08-25T04:43:35.098418+0000","last_fullsized":"2023-08-25T04:43:35.098418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:53.277068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765316+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:43:34.765316+0000","last_peered":"2023-08-25T04:43:34.765316+0000","last_clean":"2023-08-25T04:43:34.765316+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:43:34.765316+0000","last_undegraded":"2023-08-25T04:43:34.765316+0000","last_fullsized":"2023-08-25T04:43:34.765316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:04.508809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761810+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:43:34.761810+0000","last_peered":"2023-08-25T04:43:34.761810+0000","last_clean":"2023-08-25T04:43:34.761810+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:43:34.761810+0000","last_undegraded":"2023-08-25T04:43:34.761810+0000","last_fullsized":"2023-08-25T04:43:34.761810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:10:02.382799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770010+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:43:34.770010+0000","last_peered":"2023-08-25T04:43:34.770010+0000","last_clean":"2023-08-25T04:43:34.770010+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:43:34.770010+0000","last_undegraded":"2023-08-25T04:43:34.770010+0000","last_fullsized":"2023-08-25T04:43:34.770010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:00:34.306505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769760+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:43:34.769760+0000","last_peered":"2023-08-25T04:43:34.769760+0000","last_clean":"2023-08-25T04:43:34.769760+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:43:34.769760+0000","last_undegraded":"2023-08-25T04:43:34.769760+0000","last_fullsized":"2023-08-25T04:43:34.769760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:23.911843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761620+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:43:34.761620+0000","last_peered":"2023-08-25T04:43:34.761620+0000","last_clean":"2023-08-25T04:43:34.761620+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:43:34.761620+0000","last_undegraded":"2023-08-25T04:43:34.761620+0000","last_fullsized":"2023-08-25T04:43:34.761620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:30.639568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765547+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:43:34.765547+0000","last_peered":"2023-08-25T04:43:34.765547+0000","last_clean":"2023-08-25T04:43:34.765547+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:43:34.765547+0000","last_undegraded":"2023-08-25T04:43:34.765547+0000","last_fullsized":"2023-08-25T04:43:34.765547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:24.659622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100513+0000","last_change":"2023-08-25T04:43:35.100513+0000","last_active":"2023-08-25T04:43:35.100513+0000","last_peered":"2023-08-25T04:43:35.100513+0000","last_clean":"2023-08-25T04:43:35.100513+0000","last_became_active":"2023-08-25T04:43:35.100067+0000","last_became_peered":"2023-08-25T04:43:35.100067+0000","last_unstale":"2023-08-25T04:43:35.100513+0000","last_undegraded":"2023-08-25T04:43:35.100513+0000","last_fullsized":"2023-08-25T04:43:35.100513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:23:03.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765693+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:43:34.765693+0000","last_peered":"2023-08-25T04:43:34.765693+0000","last_clean":"2023-08-25T04:43:34.765693+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:43:34.765693+0000","last_undegraded":"2023-08-25T04:43:34.765693+0000","last_fullsized":"2023-08-25T04:43:34.765693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:38:39.362078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765706+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:43:34.765706+0000","last_peered":"2023-08-25T04:43:34.765706+0000","last_clean":"2023-08-25T04:43:34.765706+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:43:34.765706+0000","last_undegraded":"2023-08-25T04:43:34.765706+0000","last_fullsized":"2023-08-25T04:43:34.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:25.041276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761550+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:43:34.761550+0000","last_peered":"2023-08-25T04:43:34.761550+0000","last_clean":"2023-08-25T04:43:34.761550+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:43:34.761550+0000","last_undegraded":"2023-08-25T04:43:34.761550+0000","last_fullsized":"2023-08-25T04:43:34.761550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:25.771337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765423+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:43:34.765423+0000","last_peered":"2023-08-25T04:43:34.765423+0000","last_clean":"2023-08-25T04:43:34.765423+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:43:34.765423+0000","last_undegraded":"2023-08-25T04:43:34.765423+0000","last_fullsized":"2023-08-25T04:43:34.765423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:27.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098820+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:43:35.098820+0000","last_peered":"2023-08-25T04:43:35.098820+0000","last_clean":"2023-08-25T04:43:35.098820+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:43:35.098820+0000","last_undegraded":"2023-08-25T04:43:35.098820+0000","last_fullsized":"2023-08-25T04:43:35.098820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:20.999420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761408+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:43:34.761408+0000","last_peered":"2023-08-25T04:43:34.761408+0000","last_clean":"2023-08-25T04:43:34.761408+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:43:34.761408+0000","last_undegraded":"2023-08-25T04:43:34.761408+0000","last_fullsized":"2023-08-25T04:43:34.761408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:14.562518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764854+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:43:34.764854+0000","last_peered":"2023-08-25T04:43:34.764854+0000","last_clean":"2023-08-25T04:43:34.764854+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:43:34.764854+0000","last_undegraded":"2023-08-25T04:43:34.764854+0000","last_fullsized":"2023-08-25T04:43:34.764854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:27.603878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099267+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:43:35.099267+0000","last_peered":"2023-08-25T04:43:35.099267+0000","last_clean":"2023-08-25T04:43:35.099267+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:43:35.099267+0000","last_undegraded":"2023-08-25T04:43:35.099267+0000","last_fullsized":"2023-08-25T04:43:35.099267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:28.457166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769245+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:43:34.769245+0000","last_peered":"2023-08-25T04:43:34.769245+0000","last_clean":"2023-08-25T04:43:34.769245+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:43:34.769245+0000","last_undegraded":"2023-08-25T04:43:34.769245+0000","last_fullsized":"2023-08-25T04:43:34.769245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:47.099556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":127,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.695688+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:44:46.695688+0000","last_peered":"2023-08-25T04:44:46.695688+0000","last_clean":"2023-08-25T04:44:46.695688+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:44:46.695688+0000","last_undegraded":"2023-08-25T04:44:46.695688+0000","last_fullsized":"2023-08-25T04:44:46.695688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:02.879296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765116+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:43:34.765116+0000","last_peered":"2023-08-25T04:43:34.765116+0000","last_clean":"2023-08-25T04:43:34.765116+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:43:34.765116+0000","last_undegraded":"2023-08-25T04:43:34.765116+0000","last_fullsized":"2023-08-25T04:43:34.765116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:18.764324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769440+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:43:34.769440+0000","last_peered":"2023-08-25T04:43:34.769440+0000","last_clean":"2023-08-25T04:43:34.769440+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:43:34.769440+0000","last_undegraded":"2023-08-25T04:43:34.769440+0000","last_fullsized":"2023-08-25T04:43:34.769440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:38.395114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764497+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:43:34.764497+0000","last_peered":"2023-08-25T04:43:34.764497+0000","last_clean":"2023-08-25T04:43:34.764497+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:43:34.764497+0000","last_undegraded":"2023-08-25T04:43:34.764497+0000","last_fullsized":"2023-08-25T04:43:34.764497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:29:44.193254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099118+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:43:35.099118+0000","last_peered":"2023-08-25T04:43:35.099118+0000","last_clean":"2023-08-25T04:43:35.099118+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:43:35.099118+0000","last_undegraded":"2023-08-25T04:43:35.099118+0000","last_fullsized":"2023-08-25T04:43:35.099118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.013041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765157+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:43:34.765157+0000","last_peered":"2023-08-25T04:43:34.765157+0000","last_clean":"2023-08-25T04:43:34.765157+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:43:34.765157+0000","last_undegraded":"2023-08-25T04:43:34.765157+0000","last_fullsized":"2023-08-25T04:43:34.765157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:14.536121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098445+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:43:35.098445+0000","last_peered":"2023-08-25T04:43:35.098445+0000","last_clean":"2023-08-25T04:43:35.098445+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:43:35.098445+0000","last_undegraded":"2023-08-25T04:43:35.098445+0000","last_fullsized":"2023-08-25T04:43:35.098445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:45.583226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764897+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:43:34.764897+0000","last_peered":"2023-08-25T04:43:34.764897+0000","last_clean":"2023-08-25T04:43:34.764897+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:43:34.764897+0000","last_undegraded":"2023-08-25T04:43:34.764897+0000","last_fullsized":"2023-08-25T04:43:34.764897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:04.925252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764614+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:43:34.764614+0000","last_peered":"2023-08-25T04:43:34.764614+0000","last_clean":"2023-08-25T04:43:34.764614+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:43:34.764614+0000","last_undegraded":"2023-08-25T04:43:34.764614+0000","last_fullsized":"2023-08-25T04:43:34.764614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:24.431309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098697+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:43:35.098697+0000","last_peered":"2023-08-25T04:43:35.098697+0000","last_clean":"2023-08-25T04:43:35.098697+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:43:35.098697+0000","last_undegraded":"2023-08-25T04:43:35.098697+0000","last_fullsized":"2023-08-25T04:43:35.098697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:26.668766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761101+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:43:34.761101+0000","last_peered":"2023-08-25T04:43:34.761101+0000","last_clean":"2023-08-25T04:43:34.761101+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:43:34.761101+0000","last_undegraded":"2023-08-25T04:43:34.761101+0000","last_fullsized":"2023-08-25T04:43:34.761101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:51.398185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765545+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:43:34.765545+0000","last_peered":"2023-08-25T04:43:34.765545+0000","last_clean":"2023-08-25T04:43:34.765545+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:43:34.765545+0000","last_undegraded":"2023-08-25T04:43:34.765545+0000","last_fullsized":"2023-08-25T04:43:34.765545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:28.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:47.696571+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:44:47.696571+0000","last_peered":"2023-08-25T04:44:47.696571+0000","last_clean":"2023-08-25T04:44:47.696571+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:44:47.696571+0000","last_undegraded":"2023-08-25T04:44:47.696571+0000","last_fullsized":"2023-08-25T04:44:47.696571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:44.974722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099154+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:43:35.099154+0000","last_peered":"2023-08-25T04:43:35.099154+0000","last_clean":"2023-08-25T04:43:35.099154+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:43:35.099154+0000","last_undegraded":"2023-08-25T04:43:35.099154+0000","last_fullsized":"2023-08-25T04:43:35.099154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:07.872117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761225+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:43:34.761225+0000","last_peered":"2023-08-25T04:43:34.761225+0000","last_clean":"2023-08-25T04:43:34.761225+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:43:34.761225+0000","last_undegraded":"2023-08-25T04:43:34.761225+0000","last_fullsized":"2023-08-25T04:43:34.761225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:18.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923185+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:43:34.923185+0000","last_peered":"2023-08-25T04:43:34.923185+0000","last_clean":"2023-08-25T04:43:34.923185+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:43:34.923185+0000","last_undegraded":"2023-08-25T04:43:34.923185+0000","last_fullsized":"2023-08-25T04:43:34.923185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:14.764732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766754+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:43:34.766754+0000","last_peered":"2023-08-25T04:43:34.766754+0000","last_clean":"2023-08-25T04:43:34.766754+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:43:34.766754+0000","last_undegraded":"2023-08-25T04:43:34.766754+0000","last_fullsized":"2023-08-25T04:43:34.766754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:01.393459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100106+0000","last_change":"2023-08-25T04:43:35.100106+0000","last_active":"2023-08-25T04:43:35.100106+0000","last_peered":"2023-08-25T04:43:35.100106+0000","last_clean":"2023-08-25T04:43:35.100106+0000","last_became_active":"2023-08-25T04:43:35.099546+0000","last_became_peered":"2023-08-25T04:43:35.099546+0000","last_unstale":"2023-08-25T04:43:35.100106+0000","last_undegraded":"2023-08-25T04:43:35.100106+0000","last_fullsized":"2023-08-25T04:43:35.100106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:25.055771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099268+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:43:35.099268+0000","last_peered":"2023-08-25T04:43:35.099268+0000","last_clean":"2023-08-25T04:43:35.099268+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:43:35.099268+0000","last_undegraded":"2023-08-25T04:43:35.099268+0000","last_fullsized":"2023-08-25T04:43:35.099268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:54.567632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098918+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:43:35.098918+0000","last_peered":"2023-08-25T04:43:35.098918+0000","last_clean":"2023-08-25T04:43:35.098918+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:43:35.098918+0000","last_undegraded":"2023-08-25T04:43:35.098918+0000","last_fullsized":"2023-08-25T04:43:35.098918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:47.455399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761107+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:43:34.761107+0000","last_peered":"2023-08-25T04:43:34.761107+0000","last_clean":"2023-08-25T04:43:34.761107+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:43:34.761107+0000","last_undegraded":"2023-08-25T04:43:34.761107+0000","last_fullsized":"2023-08-25T04:43:34.761107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:27:18.610077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:48.697368+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:44:48.697368+0000","last_peered":"2023-08-25T04:44:48.697368+0000","last_clean":"2023-08-25T04:44:48.697368+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:44:48.697368+0000","last_undegraded":"2023-08-25T04:44:48.697368+0000","last_fullsized":"2023-08-25T04:44:48.697368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:00.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":188,"num_read_kb":98,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765074+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:43:34.765074+0000","last_peered":"2023-08-25T04:43:34.765074+0000","last_clean":"2023-08-25T04:43:34.765074+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:43:34.765074+0000","last_undegraded":"2023-08-25T04:43:34.765074+0000","last_fullsized":"2023-08-25T04:43:34.765074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:07.231855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099639+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:43:35.099639+0000","last_peered":"2023-08-25T04:43:35.099639+0000","last_clean":"2023-08-25T04:43:35.099639+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:43:35.099639+0000","last_undegraded":"2023-08-25T04:43:35.099639+0000","last_fullsized":"2023-08-25T04:43:35.099639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:46:34.120949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769705+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:43:34.769705+0000","last_peered":"2023-08-25T04:43:34.769705+0000","last_clean":"2023-08-25T04:43:34.769705+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:43:34.769705+0000","last_undegraded":"2023-08-25T04:43:34.769705+0000","last_fullsized":"2023-08-25T04:43:34.769705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:00.803708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098704+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:43:35.098704+0000","last_peered":"2023-08-25T04:43:35.098704+0000","last_clean":"2023-08-25T04:43:35.098704+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:43:35.098704+0000","last_undegraded":"2023-08-25T04:43:35.098704+0000","last_fullsized":"2023-08-25T04:43:35.098704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:19:57.215688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.767465+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:43:34.767465+0000","last_peered":"2023-08-25T04:43:34.767465+0000","last_clean":"2023-08-25T04:43:34.767465+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:43:34.767465+0000","last_undegraded":"2023-08-25T04:43:34.767465+0000","last_fullsized":"2023-08-25T04:43:34.767465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:22.379000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098949+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:43:35.098949+0000","last_peered":"2023-08-25T04:43:35.098949+0000","last_clean":"2023-08-25T04:43:35.098949+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:43:35.098949+0000","last_undegraded":"2023-08-25T04:43:35.098949+0000","last_fullsized":"2023-08-25T04:43:35.098949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:38.602930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923061+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:43:34.923061+0000","last_peered":"2023-08-25T04:43:34.923061+0000","last_clean":"2023-08-25T04:43:34.923061+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:43:34.923061+0000","last_undegraded":"2023-08-25T04:43:34.923061+0000","last_fullsized":"2023-08-25T04:43:34.923061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:57.719232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761321+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:43:34.761321+0000","last_peered":"2023-08-25T04:43:34.761321+0000","last_clean":"2023-08-25T04:43:34.761321+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:43:34.761321+0000","last_undegraded":"2023-08-25T04:43:34.761321+0000","last_fullsized":"2023-08-25T04:43:34.761321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:45.122188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.718684+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:44:46.718684+0000","last_peered":"2023-08-25T04:44:46.718684+0000","last_clean":"2023-08-25T04:44:46.718684+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:44:46.718684+0000","last_undegraded":"2023-08-25T04:44:46.718684+0000","last_fullsized":"2023-08-25T04:44:46.718684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:03:39.958808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922765+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:43:34.922765+0000","last_peered":"2023-08-25T04:43:34.922765+0000","last_clean":"2023-08-25T04:43:34.922765+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:43:34.922765+0000","last_undegraded":"2023-08-25T04:43:34.922765+0000","last_fullsized":"2023-08-25T04:43:34.922765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:22.482680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099199+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:43:35.099199+0000","last_peered":"2023-08-25T04:43:35.099199+0000","last_clean":"2023-08-25T04:43:35.099199+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:43:35.099199+0000","last_undegraded":"2023-08-25T04:43:35.099199+0000","last_fullsized":"2023-08-25T04:43:35.099199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:46.918540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765654+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:43:34.765654+0000","last_peered":"2023-08-25T04:43:34.765654+0000","last_clean":"2023-08-25T04:43:34.765654+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:43:34.765654+0000","last_undegraded":"2023-08-25T04:43:34.765654+0000","last_fullsized":"2023-08-25T04:43:34.765654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:28.897000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:02.027948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098592+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:43:35.098592+0000","last_peered":"2023-08-25T04:43:35.098592+0000","last_clean":"2023-08-25T04:43:35.098592+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:43:35.098592+0000","last_undegraded":"2023-08-25T04:43:35.098592+0000","last_fullsized":"2023-08-25T04:43:35.098592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:19:33.883508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764534+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:43:34.764534+0000","last_peered":"2023-08-25T04:43:34.764534+0000","last_clean":"2023-08-25T04:43:34.764534+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:43:34.764534+0000","last_undegraded":"2023-08-25T04:43:34.764534+0000","last_fullsized":"2023-08-25T04:43:34.764534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:04:26.603958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766834+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:43:34.766834+0000","last_peered":"2023-08-25T04:43:34.766834+0000","last_clean":"2023-08-25T04:43:34.766834+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:43:34.766834+0000","last_undegraded":"2023-08-25T04:43:34.766834+0000","last_fullsized":"2023-08-25T04:43:34.766834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:35:18.420398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764942+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:43:34.764942+0000","last_peered":"2023-08-25T04:43:34.764942+0000","last_clean":"2023-08-25T04:43:34.764942+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:43:34.764942+0000","last_undegraded":"2023-08-25T04:43:34.764942+0000","last_fullsized":"2023-08-25T04:43:34.764942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:31.579981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761767+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:43:34.761767+0000","last_peered":"2023-08-25T04:43:34.761767+0000","last_clean":"2023-08-25T04:43:34.761767+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:43:34.761767+0000","last_undegraded":"2023-08-25T04:43:34.761767+0000","last_fullsized":"2023-08-25T04:43:34.761767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:32.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764777+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:43:34.764777+0000","last_peered":"2023-08-25T04:43:34.764777+0000","last_clean":"2023-08-25T04:43:34.764777+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:43:34.764777+0000","last_undegraded":"2023-08-25T04:43:34.764777+0000","last_fullsized":"2023-08-25T04:43:34.764777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:25:29.024128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099012+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:43:35.099012+0000","last_peered":"2023-08-25T04:43:35.099012+0000","last_clean":"2023-08-25T04:43:35.099012+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:43:35.099012+0000","last_undegraded":"2023-08-25T04:43:35.099012+0000","last_fullsized":"2023-08-25T04:43:35.099012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:17.119675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922819+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:43:34.922819+0000","last_peered":"2023-08-25T04:43:34.922819+0000","last_clean":"2023-08-25T04:43:34.922819+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:43:34.922819+0000","last_undegraded":"2023-08-25T04:43:34.922819+0000","last_fullsized":"2023-08-25T04:43:34.922819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:42.375965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769924+0000","last_change":"2023-08-25T04:43:34.769924+0000","last_active":"2023-08-25T04:43:34.769924+0000","last_peered":"2023-08-25T04:43:34.769924+0000","last_clean":"2023-08-25T04:43:34.769924+0000","last_became_active":"2023-08-25T04:43:34.769575+0000","last_became_peered":"2023-08-25T04:43:34.769575+0000","last_unstale":"2023-08-25T04:43:34.769924+0000","last_undegraded":"2023-08-25T04:43:34.769924+0000","last_fullsized":"2023-08-25T04:43:34.769924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:32.013614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099091+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:43:35.099091+0000","last_peered":"2023-08-25T04:43:35.099091+0000","last_clean":"2023-08-25T04:43:35.099091+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:43:35.099091+0000","last_undegraded":"2023-08-25T04:43:35.099091+0000","last_fullsized":"2023-08-25T04:43:35.099091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:32:02.438964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.772835+0000","last_change":"2023-08-25T04:43:34.771233+0000","last_active":"2023-08-25T04:43:34.772835+0000","last_peered":"2023-08-25T04:43:34.772835+0000","last_clean":"2023-08-25T04:43:34.772835+0000","last_became_active":"2023-08-25T04:43:34.770887+0000","last_became_peered":"2023-08-25T04:43:34.770887+0000","last_unstale":"2023-08-25T04:43:34.772835+0000","last_undegraded":"2023-08-25T04:43:34.772835+0000","last_fullsized":"2023-08-25T04:43:34.772835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:08.119078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":122,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.717700+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:44:46.717700+0000","last_peered":"2023-08-25T04:44:46.717700+0000","last_clean":"2023-08-25T04:44:46.717700+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:44:46.717700+0000","last_undegraded":"2023-08-25T04:44:46.717700+0000","last_fullsized":"2023-08-25T04:44:46.717700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:43.271628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922975+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:43:34.922975+0000","last_peered":"2023-08-25T04:43:34.922975+0000","last_clean":"2023-08-25T04:43:34.922975+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:43:34.922975+0000","last_undegraded":"2023-08-25T04:43:34.922975+0000","last_fullsized":"2023-08-25T04:43:34.922975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:04.252263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764688+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:43:34.764688+0000","last_peered":"2023-08-25T04:43:34.764688+0000","last_clean":"2023-08-25T04:43:34.764688+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:43:34.764688+0000","last_undegraded":"2023-08-25T04:43:34.764688+0000","last_fullsized":"2023-08-25T04:43:34.764688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:33:20.016878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761743+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:43:34.761743+0000","last_peered":"2023-08-25T04:43:34.761743+0000","last_clean":"2023-08-25T04:43:34.761743+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:43:34.761743+0000","last_undegraded":"2023-08-25T04:43:34.761743+0000","last_fullsized":"2023-08-25T04:43:34.761743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:10.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922646+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:43:34.922646+0000","last_peered":"2023-08-25T04:43:34.922646+0000","last_clean":"2023-08-25T04:43:34.922646+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:43:34.922646+0000","last_undegraded":"2023-08-25T04:43:34.922646+0000","last_fullsized":"2023-08-25T04:43:34.922646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:20.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099295+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:43:35.099295+0000","last_peered":"2023-08-25T04:43:35.099295+0000","last_clean":"2023-08-25T04:43:35.099295+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:43:35.099295+0000","last_undegraded":"2023-08-25T04:43:35.099295+0000","last_fullsized":"2023-08-25T04:43:35.099295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.332881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098261+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:43:35.098261+0000","last_peered":"2023-08-25T04:43:35.098261+0000","last_clean":"2023-08-25T04:43:35.098261+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:43:35.098261+0000","last_undegraded":"2023-08-25T04:43:35.098261+0000","last_fullsized":"2023-08-25T04:43:35.098261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:51:31.902430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922785+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:43:34.922785+0000","last_peered":"2023-08-25T04:43:34.922785+0000","last_clean":"2023-08-25T04:43:34.922785+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:43:34.922785+0000","last_undegraded":"2023-08-25T04:43:34.922785+0000","last_fullsized":"2023-08-25T04:43:34.922785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:47.704937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765952+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:43:34.765952+0000","last_peered":"2023-08-25T04:43:34.765952+0000","last_clean":"2023-08-25T04:43:34.765952+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:43:34.765952+0000","last_undegraded":"2023-08-25T04:43:34.765952+0000","last_fullsized":"2023-08-25T04:43:34.765952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:38:36.745026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765218+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:43:34.765218+0000","last_peered":"2023-08-25T04:43:34.765218+0000","last_clean":"2023-08-25T04:43:34.765218+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:43:34.765218+0000","last_undegraded":"2023-08-25T04:43:34.765218+0000","last_fullsized":"2023-08-25T04:43:34.765218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:15:24.598228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:14:26.761777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764760+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:43:34.764760+0000","last_peered":"2023-08-25T04:43:34.764760+0000","last_clean":"2023-08-25T04:43:34.764760+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:43:34.764760+0000","last_undegraded":"2023-08-25T04:43:34.764760+0000","last_fullsized":"2023-08-25T04:43:34.764760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:23.436529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.777294+0000","last_change":"2023-08-25T04:43:34.777294+0000","last_active":"2023-08-25T04:43:34.777294+0000","last_peered":"2023-08-25T04:43:34.777294+0000","last_clean":"2023-08-25T04:43:34.777294+0000","last_became_active":"2023-08-25T04:43:34.768036+0000","last_became_peered":"2023-08-25T04:43:34.768036+0000","last_unstale":"2023-08-25T04:43:34.777294+0000","last_undegraded":"2023-08-25T04:43:34.777294+0000","last_fullsized":"2023-08-25T04:43:34.777294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:52.521174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765705+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:43:34.765705+0000","last_peered":"2023-08-25T04:43:34.765705+0000","last_clean":"2023-08-25T04:43:34.765705+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:43:34.765705+0000","last_undegraded":"2023-08-25T04:43:34.765705+0000","last_fullsized":"2023-08-25T04:43:34.765705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:34.577696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768990+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:43:34.768990+0000","last_peered":"2023-08-25T04:43:34.768990+0000","last_clean":"2023-08-25T04:43:34.768990+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:43:34.768990+0000","last_undegraded":"2023-08-25T04:43:34.768990+0000","last_fullsized":"2023-08-25T04:43:34.768990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:09.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761921+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:43:34.761921+0000","last_peered":"2023-08-25T04:43:34.761921+0000","last_clean":"2023-08-25T04:43:34.761921+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:43:34.761921+0000","last_undegraded":"2023-08-25T04:43:34.761921+0000","last_fullsized":"2023-08-25T04:43:34.761921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:16.228950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764889+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:43:34.764889+0000","last_peered":"2023-08-25T04:43:34.764889+0000","last_clean":"2023-08-25T04:43:34.764889+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:43:34.764889+0000","last_undegraded":"2023-08-25T04:43:34.764889+0000","last_fullsized":"2023-08-25T04:43:34.764889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:14.043346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768669+0000","last_change":"2023-08-25T04:43:34.768669+0000","last_active":"2023-08-25T04:43:34.768669+0000","last_peered":"2023-08-25T04:43:34.768669+0000","last_clean":"2023-08-25T04:43:34.768669+0000","last_became_active":"2023-08-25T04:43:34.768275+0000","last_became_peered":"2023-08-25T04:43:34.768275+0000","last_unstale":"2023-08-25T04:43:34.768669+0000","last_undegraded":"2023-08-25T04:43:34.768669+0000","last_fullsized":"2023-08-25T04:43:34.768669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:39:53.898292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,11],"acting":[4,8,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098506+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:43:35.098506+0000","last_peered":"2023-08-25T04:43:35.098506+0000","last_clean":"2023-08-25T04:43:35.098506+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:43:35.098506+0000","last_undegraded":"2023-08-25T04:43:35.098506+0000","last_fullsized":"2023-08-25T04:43:35.098506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:19.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764296+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:43:34.764296+0000","last_peered":"2023-08-25T04:43:34.764296+0000","last_clean":"2023-08-25T04:43:34.764296+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:43:34.764296+0000","last_undegraded":"2023-08-25T04:43:34.764296+0000","last_fullsized":"2023-08-25T04:43:34.764296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:11:04.726743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761549+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:43:34.761549+0000","last_peered":"2023-08-25T04:43:34.761549+0000","last_clean":"2023-08-25T04:43:34.761549+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:43:34.761549+0000","last_undegraded":"2023-08-25T04:43:34.761549+0000","last_fullsized":"2023-08-25T04:43:34.761549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:48.355591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765742+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:43:34.765742+0000","last_peered":"2023-08-25T04:43:34.765742+0000","last_clean":"2023-08-25T04:43:34.765742+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:43:34.765742+0000","last_undegraded":"2023-08-25T04:43:34.765742+0000","last_fullsized":"2023-08-25T04:43:34.765742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:04.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766384+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:43:34.766384+0000","last_peered":"2023-08-25T04:43:34.766384+0000","last_clean":"2023-08-25T04:43:34.766384+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:43:34.766384+0000","last_undegraded":"2023-08-25T04:43:34.766384+0000","last_fullsized":"2023-08-25T04:43:34.766384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:02:40.882696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761618+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:43:34.761618+0000","last_peered":"2023-08-25T04:43:34.761618+0000","last_clean":"2023-08-25T04:43:34.761618+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:43:34.761618+0000","last_undegraded":"2023-08-25T04:43:34.761618+0000","last_fullsized":"2023-08-25T04:43:34.761618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:23.594133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.779656+0000","last_change":"2023-08-25T04:43:34.779656+0000","last_active":"2023-08-25T04:43:34.779656+0000","last_peered":"2023-08-25T04:43:34.779656+0000","last_clean":"2023-08-25T04:43:34.779656+0000","last_became_active":"2023-08-25T04:43:34.770821+0000","last_became_peered":"2023-08-25T04:43:34.770821+0000","last_unstale":"2023-08-25T04:43:34.779656+0000","last_undegraded":"2023-08-25T04:43:34.779656+0000","last_fullsized":"2023-08-25T04:43:34.779656+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:08:31.402768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766129+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:43:34.766129+0000","last_peered":"2023-08-25T04:43:34.766129+0000","last_clean":"2023-08-25T04:43:34.766129+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:43:34.766129+0000","last_undegraded":"2023-08-25T04:43:34.766129+0000","last_fullsized":"2023-08-25T04:43:34.766129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:18.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761763+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:43:34.761763+0000","last_peered":"2023-08-25T04:43:34.761763+0000","last_clean":"2023-08-25T04:43:34.761763+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:43:34.761763+0000","last_undegraded":"2023-08-25T04:43:34.761763+0000","last_fullsized":"2023-08-25T04:43:34.761763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:14.914419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765951+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:43:34.765951+0000","last_peered":"2023-08-25T04:43:34.765951+0000","last_clean":"2023-08-25T04:43:34.765951+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:43:34.765951+0000","last_undegraded":"2023-08-25T04:43:34.765951+0000","last_fullsized":"2023-08-25T04:43:34.765951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:07.289831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922875+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:43:34.922875+0000","last_peered":"2023-08-25T04:43:34.922875+0000","last_clean":"2023-08-25T04:43:34.922875+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:43:34.922875+0000","last_undegraded":"2023-08-25T04:43:34.922875+0000","last_fullsized":"2023-08-25T04:43:34.922875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:13.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098664+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:43:35.098664+0000","last_peered":"2023-08-25T04:43:35.098664+0000","last_clean":"2023-08-25T04:43:35.098664+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:43:35.098664+0000","last_undegraded":"2023-08-25T04:43:35.098664+0000","last_fullsized":"2023-08-25T04:43:35.098664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:02:41.742505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769588+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:43:34.769588+0000","last_peered":"2023-08-25T04:43:34.769588+0000","last_clean":"2023-08-25T04:43:34.769588+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:43:34.769588+0000","last_undegraded":"2023-08-25T04:43:34.769588+0000","last_fullsized":"2023-08-25T04:43:34.769588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:56.254087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923325+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:43:34.923325+0000","last_peered":"2023-08-25T04:43:34.923325+0000","last_clean":"2023-08-25T04:43:34.923325+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:43:34.923325+0000","last_undegraded":"2023-08-25T04:43:34.923325+0000","last_fullsized":"2023-08-25T04:43:34.923325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:27.560915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764860+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:43:34.764860+0000","last_peered":"2023-08-25T04:43:34.764860+0000","last_clean":"2023-08-25T04:43:34.764860+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:43:34.764860+0000","last_undegraded":"2023-08-25T04:43:34.764860+0000","last_fullsized":"2023-08-25T04:43:34.764860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:05.061144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099452+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:43:35.099452+0000","last_peered":"2023-08-25T04:43:35.099452+0000","last_clean":"2023-08-25T04:43:35.099452+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:43:35.099452+0000","last_undegraded":"2023-08-25T04:43:35.099452+0000","last_fullsized":"2023-08-25T04:43:35.099452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:28.319336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923060+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:43:34.923060+0000","last_peered":"2023-08-25T04:43:34.923060+0000","last_clean":"2023-08-25T04:43:34.923060+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:43:34.923060+0000","last_undegraded":"2023-08-25T04:43:34.923060+0000","last_fullsized":"2023-08-25T04:43:34.923060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:25.511917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098448+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:43:35.098448+0000","last_peered":"2023-08-25T04:43:35.098448+0000","last_clean":"2023-08-25T04:43:35.098448+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:43:35.098448+0000","last_undegraded":"2023-08-25T04:43:35.098448+0000","last_fullsized":"2023-08-25T04:43:35.098448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:15.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099187+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:43:35.099187+0000","last_peered":"2023-08-25T04:43:35.099187+0000","last_clean":"2023-08-25T04:43:35.099187+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:43:35.099187+0000","last_undegraded":"2023-08-25T04:43:35.099187+0000","last_fullsized":"2023-08-25T04:43:35.099187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:00.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923329+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:43:34.923329+0000","last_peered":"2023-08-25T04:43:34.923329+0000","last_clean":"2023-08-25T04:43:34.923329+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:43:34.923329+0000","last_undegraded":"2023-08-25T04:43:34.923329+0000","last_fullsized":"2023-08-25T04:43:34.923329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:03.933897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766391+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:43:34.766391+0000","last_peered":"2023-08-25T04:43:34.766391+0000","last_clean":"2023-08-25T04:43:34.766391+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:43:34.766391+0000","last_undegraded":"2023-08-25T04:43:34.766391+0000","last_fullsized":"2023-08-25T04:43:34.766391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:42.835852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769164+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:43:34.769164+0000","last_peered":"2023-08-25T04:43:34.769164+0000","last_clean":"2023-08-25T04:43:34.769164+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:43:34.769164+0000","last_undegraded":"2023-08-25T04:43:34.769164+0000","last_fullsized":"2023-08-25T04:43:34.769164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:38.408069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769152+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:43:34.769152+0000","last_peered":"2023-08-25T04:43:34.769152+0000","last_clean":"2023-08-25T04:43:34.769152+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:43:34.769152+0000","last_undegraded":"2023-08-25T04:43:34.769152+0000","last_fullsized":"2023-08-25T04:43:34.769152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:25.658427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100100+0000","last_change":"2023-08-25T04:43:35.100100+0000","last_active":"2023-08-25T04:43:35.100100+0000","last_peered":"2023-08-25T04:43:35.100100+0000","last_clean":"2023-08-25T04:43:35.100100+0000","last_became_active":"2023-08-25T04:43:35.099734+0000","last_became_peered":"2023-08-25T04:43:35.099734+0000","last_unstale":"2023-08-25T04:43:35.100100+0000","last_undegraded":"2023-08-25T04:43:35.100100+0000","last_fullsized":"2023-08-25T04:43:35.100100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:17.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099291+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:43:35.099291+0000","last_peered":"2023-08-25T04:43:35.099291+0000","last_clean":"2023-08-25T04:43:35.099291+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:43:35.099291+0000","last_undegraded":"2023-08-25T04:43:35.099291+0000","last_fullsized":"2023-08-25T04:43:35.099291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:05:39.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766561+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:43:34.766561+0000","last_peered":"2023-08-25T04:43:34.766561+0000","last_clean":"2023-08-25T04:43:34.766561+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:43:34.766561+0000","last_undegraded":"2023-08-25T04:43:34.766561+0000","last_fullsized":"2023-08-25T04:43:34.766561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:47:14.892189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770202+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:43:34.770202+0000","last_peered":"2023-08-25T04:43:34.770202+0000","last_clean":"2023-08-25T04:43:34.770202+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:43:34.770202+0000","last_undegraded":"2023-08-25T04:43:34.770202+0000","last_fullsized":"2023-08-25T04:43:34.770202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:31.428150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765901+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:43:34.765901+0000","last_peered":"2023-08-25T04:43:34.765901+0000","last_clean":"2023-08-25T04:43:34.765901+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:43:34.765901+0000","last_undegraded":"2023-08-25T04:43:34.765901+0000","last_fullsized":"2023-08-25T04:43:34.765901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:04.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:13.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765879+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:43:34.765879+0000","last_peered":"2023-08-25T04:43:34.765879+0000","last_clean":"2023-08-25T04:43:34.765879+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:43:34.765879+0000","last_undegraded":"2023-08-25T04:43:34.765879+0000","last_fullsized":"2023-08-25T04:43:34.765879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:03:35.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:15:06.086740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098646+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:43:35.098646+0000","last_peered":"2023-08-25T04:43:35.098646+0000","last_clean":"2023-08-25T04:43:35.098646+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:43:35.098646+0000","last_undegraded":"2023-08-25T04:43:35.098646+0000","last_fullsized":"2023-08-25T04:43:35.098646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:42.910594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.771119+0000","last_change":"2023-08-25T04:43:34.771119+0000","last_active":"2023-08-25T04:43:34.771119+0000","last_peered":"2023-08-25T04:43:34.771119+0000","last_clean":"2023-08-25T04:43:34.771119+0000","last_became_active":"2023-08-25T04:43:34.770717+0000","last_became_peered":"2023-08-25T04:43:34.770717+0000","last_unstale":"2023-08-25T04:43:34.771119+0000","last_undegraded":"2023-08-25T04:43:34.771119+0000","last_fullsized":"2023-08-25T04:43:34.771119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:42.596435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":576,"num_read_kb":302,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":483313,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":1.018,"15min":1.018},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90500000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70599999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71099999999999997,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.54700000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.496}]},{"osd":5,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84999999999999998,"5min":1.006,"15min":1.006},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.875,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.64800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":1.099,"15min":1.099},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.69999999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.66300000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79400000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.76300000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.85699999999999998,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.72099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97899999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72499999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1078519,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91100000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.036,"15min":1.036},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.91000000000000003,"5min":1.167,"15min":1.167},"last":0.80000000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77700000000000002},"min":{"1min":0.64500000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.72399999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841580,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483549,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.61299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.432},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.745},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84899999999999998,"5min":1.266,"15min":1.266},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70599999999999996,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.73799999999999999,"15min":0.78200000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69799999999999995,"15min":0.73299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.71899999999999997},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.628}]},{"osd":4,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80500000000000005,"15min":0.86499999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88200000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.85599999999999998},"min":{"1min":0.501,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.93799999999999994},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.87,"15min":0.92300000000000004},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.873,"15min":0.92700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.413,"15min":1.413},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.95999999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.47,"15min":1.47},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.876}]},{"osd":11,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103217,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483138,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60299999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.441,"15min":0.41999999999999998},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.746,"15min":0.746},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8730000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.871}]},{"osd":4,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.754}]},{"osd":5,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83199999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93200000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70899999999999996,"15min":0.68400000000000005},"min":{"1min":0.57899999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Aug 25 04:44:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.82599999999999996,"15min":0.78100000000000003},"min":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.9119999999999999,"5min":4.9119999999999999,"15min":4.9119999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.83599999999999997,"15min":0.78800000000000003},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.9569999999999999,"5min":3.9569999999999999,"15min":3.9569999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.61399999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.75},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97399999999999998,"5min":1.024,"15min":1.024},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86199999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074818,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.41099999999999998},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.876,"15min":0.876},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.628},"min":{"1min":0.51900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71799999999999997,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95499999999999996,"5min":0.996,"15min":0.996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.623}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.8980000000000001,"5min":3.8980000000000001,"15min":3.8980000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":0.79800000000000004},"min":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.014,"15min":1.014},"last":0.67400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.78200000000000003,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96599999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76200000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482821,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.57799999999999996,"15min":0.50900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.347,"5min":3.347,"15min":3.347},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.46899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68600000000000005,"15min":0.66100000000000003},"min":{"1min":0.53700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.745,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.69799999999999995,"15min":0.63100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.498},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.69299999999999995,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.625}]},{"osd":4,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.77700000000000002,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.502,"5min":0.442,"15min":0.442},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0329999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.80700000000000005,"15min":0.78500000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.84699999999999998,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.409,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.56299999999999994,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.1020000000000001,"5min":0.93000000000000005,"15min":0.90700000000000003},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.746}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91300000000000003,"15min":0.90600000000000003},"min":{"1min":0.68700000000000006,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.82699999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.873},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.83999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":486300,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96299999999999997,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.495}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.52600000000000002},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":1.121,"15min":1.121},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59199999999999997,"15min":0.51200000000000001},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":1.548,"15min":1.548},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.42499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.401,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.59899999999999998},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.39800000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.436},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72299999999999998,"5min":1.153,"15min":1.153},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96999999999999997,"5min":1.613,"15min":1.613},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69699999999999995,"15min":0.66400000000000003},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.2210000000000001,"5min":1.597,"15min":1.597},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.2130000000000001,"5min":1.26,"15min":1.26},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.75,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.375,"5min":11.375,"15min":11.375},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82899999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83999999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40799999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.76400000000000001,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.367,"5min":3.367,"15min":3.367},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.83499999999999996,"15min":0.79700000000000004},"min":{"1min":0.60599999999999998,"5min":0.437,"15min":0.437},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.79400000000000004,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93000000000000005,"5min":1.302,"15min":1.302},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92100000000000004,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.79500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":484209,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.67300000000000004,"15min":0.54700000000000004},"min":{"1min":0.48699999999999999,"5min":0.216,"15min":0.216},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.624,"15min":0.54600000000000004},"min":{"1min":0.51600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.53500000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95699999999999996,"5min":1.194,"15min":1.194},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.63,"15min":0.57199999999999995},"min":{"1min":0.52100000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.499}]},{"osd":4,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.69299999999999995,"15min":0.60399999999999998},"min":{"1min":0.52000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.5960000000000001,"5min":4.5960000000000001,"15min":4.5960000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.48899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.68400000000000005,"15min":0.63400000000000001},"min":{"1min":0.56999999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94799999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.028,"5min":1.365,"15min":1.365},"last":1.0029999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83399999999999996,"15min":0.82099999999999995},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.747},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.98799999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85799999999999998,"15min":0.82699999999999996},"min":{"1min":0.72099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93899999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.68799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.73399999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.087,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.96499999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":488181,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.53000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90500000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89500000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.622}]},{"osd":4,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.45100000000000001},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.45000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.61799999999999999},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96399999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63300000000000001,"15min":0.59199999999999997},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89900000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47899999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.622,"5min":0.44,"15min":0.44},"max":{"1min":0.98299999999999998,"5min":1.071,"15min":1.071},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.71299999999999997},"min":{"1min":0.59899999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.59999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80800000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.747},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83099999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483015,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.40100000000000002},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.42799999999999999},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.95099999999999996}]},{"osd":1,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.58299999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.341,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":1.53,"15min":1.53},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.573,"15min":1.573},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.628},"min":{"1min":0.52000000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.4410000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Aug 25 04:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.73999999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.077,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.747,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.337,"15min":1.337},"last":0.621}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.85899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.495,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.67800000000000005}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.88900000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.145},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1020000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66800000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.64600000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":483375,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41399999999999998,"15min":0.36199999999999999},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.753,"15min":0.753},"last":0.374},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.501},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.625,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84999999999999998,"5min":1.99,"15min":1.99},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.55700000000000005},"min":{"1min":0.53400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88200000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.67600000000000005},"min":{"1min":0.69399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84799999999999998},"min":{"1min":0.64000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0880000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.099,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.73599999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80200000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1074833,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.73299999999999998,"15min":0.59399999999999997},"min":{"1min":0.42099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":16.949999999999999,"5min":16.949999999999999,"15min":16.949999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.52600000000000002},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56499999999999995,"15min":0.50900000000000001},"min":{"1min":0.48299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.375},"min":{"1min":0.33200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70799999999999996,"5min":0.755,"15min":0.755},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.40000000000000002},"min":{"1min":0.31900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67300000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.56200000000000006},"min":{"1min":0.51700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.876,"5min":0.88,"15min":0.88},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.66600000000000004,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.57299999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.60799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.88200000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.071},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.79900000000000004,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":5.8479999999999999,"5min":5.8479999999999999,"15min":5.8479999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.61599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.72999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:44:51.292 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-08-25T04:44:51.292 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T04:44:51.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 -- 172.21.15.145:0/4033257506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf781024c0 msgr2=0x7faf78108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 --2- 172.21.15.145:0/4033257506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf781024c0 0x7faf78108d70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7faf7000b0d0 tx=0x7faf7000b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 -- 172.21.15.145:0/4033257506 shutdown_connections 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 --2- 172.21.15.145:0/4033257506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf781024c0 0x7faf78108d70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 --2- 172.21.15.145:0/4033257506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 0x7faf78100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 --2- 172.21.15.145:0/4033257506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf78101b20 0x7faf78101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 -- 172.21.15.145:0/4033257506 >> 172.21.15.145:0/4033257506 conn(0x7faf780fbe60 msgr2=0x7faf780fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:51.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 -- 172.21.15.145:0/4033257506 shutdown_connections 2023-08-25T04:44:51.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.519+0000 7faf7ddff700 1 -- 172.21.15.145:0/4033257506 wait complete. 2023-08-25T04:44:51.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 Processor -- start 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 -- start start 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 0x7faf78198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf781024c0 0x7faf7819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a0480 con 0x7faf78101b20 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a05c0 con 0x7faf78100930 2023-08-25T04:44:51.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.520+0000 7faf7ddff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a0750 con 0x7faf781024c0 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54686/0 (socket says 172.21.15.145:54686) 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 -- 172.21.15.145:0/693275649 learned_addr learned my addr 172.21.15.145:0/693275649 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf777fe700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 0x7faf78198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf77fff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf781024c0 0x7faf7819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 -- 172.21.15.145:0/693275649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf781024c0 msgr2=0x7faf7819feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf781024c0 0x7faf7819feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 -- 172.21.15.145:0/693275649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 msgr2=0x7faf78198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 0x7faf78198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 -- 172.21.15.145:0/693275649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009520 con 0x7faf78101b20 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf76ffd700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00eb30 tx=0x7faf6c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.521+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c009880 con 0x7faf78101b20 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.522+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf70009d00 con 0x7faf78101b20 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.522+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6c0044e0 con 0x7faf78101b20 2023-08-25T04:44:51.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.522+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c003e90 con 0x7faf78101b20 2023-08-25T04:44:51.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.522+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf781a0e00 con 0x7faf78101b20 2023-08-25T04:44:51.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.523+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7810c1a0 con 0x7faf78101b20 2023-08-25T04:44:51.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.525+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7faf6c004000 con 0x7faf78101b20 2023-08-25T04:44:51.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.526+0000 7faf74ff9700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 0x7faf600a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:51.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.526+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faf6c014070 con 0x7faf78101b20 2023-08-25T04:44:51.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.527+0000 7faf777fe700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 0x7faf600a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:51.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.527+0000 7faf777fe700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 0x7faf600a2460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf78101980 tx=0x7faf68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:51.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.529+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faf6c095570 con 0x7faf78101b20 2023-08-25T04:44:51.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.731+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf7810c3c0 con 0x7faf6009ffa0 2023-08-25T04:44:51.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.743+0000 7faf74ff9700 1 -- 172.21.15.145:0/693275649 <== mgr.14150 v2:172.21.15.145:6800/891678096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427232 (secure 0 0 0) 0x7faf7810c3c0 con 0x7faf6009ffa0 2023-08-25T04:44:51.749 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 msgr2=0x7faf600a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 0x7faf600a2460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf78101980 tx=0x7faf68009260 comp rx=0 tx=0).stop 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 msgr2=0x7faf78198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00eb30 tx=0x7faf6c00ee40 comp rx=0 tx=0).stop 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 shutdown_connections 2023-08-25T04:44:51.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7faf6009ffa0 0x7faf600a2460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf781024c0 0x7faf7819feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf78100930 0x7faf78198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 --2- 172.21.15.145:0/693275649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf78101b20 0x7faf78198e40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.748+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 >> 172.21.15.145:0/693275649 conn(0x7faf780fbe60 msgr2=0x7faf78107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.749+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 shutdown_connections 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:51.749+0000 7faf7ddff700 1 -- 172.21.15.145:0/693275649 wait complete. 2023-08-25T04:44:51.753 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-08-25T04:44:51.769 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-08-25T04:44:51.715377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598078,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":712,"num_read_kb":421,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":11652,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":11931648,"data_stored":7586071,"data_compressed":318375,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003567"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765465+0000","last_change":"2023-08-25T04:42:50.329666+0000","last_active":"2023-08-25T04:43:34.765465+0000","last_peered":"2023-08-25T04:43:34.765465+0000","last_clean":"2023-08-25T04:43:34.765465+0000","last_became_active":"2023-08-25T04:42:50.329354+0000","last_became_peered":"2023-08-25T04:42:50.329354+0000","last_unstale":"2023-08-25T04:43:34.765465+0000","last_undegraded":"2023-08-25T04:43:34.765465+0000","last_fullsized":"2023-08-25T04:43:34.765465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:59.835651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764768+0000","last_change":"2023-08-25T04:42:50.328825+0000","last_active":"2023-08-25T04:43:34.764768+0000","last_peered":"2023-08-25T04:43:34.764768+0000","last_clean":"2023-08-25T04:43:34.764768+0000","last_became_active":"2023-08-25T04:42:50.328483+0000","last_became_peered":"2023-08-25T04:42:50.328483+0000","last_unstale":"2023-08-25T04:43:34.764768+0000","last_undegraded":"2023-08-25T04:43:34.764768+0000","last_fullsized":"2023-08-25T04:43:34.764768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:56:09.848052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766305+0000","last_change":"2023-08-25T04:42:50.331982+0000","last_active":"2023-08-25T04:43:34.766305+0000","last_peered":"2023-08-25T04:43:34.766305+0000","last_clean":"2023-08-25T04:43:34.766305+0000","last_became_active":"2023-08-25T04:42:50.331751+0000","last_became_peered":"2023-08-25T04:42:50.331751+0000","last_unstale":"2023-08-25T04:43:34.766305+0000","last_undegraded":"2023-08-25T04:43:34.766305+0000","last_fullsized":"2023-08-25T04:43:34.766305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:56.683325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762343+0000","last_change":"2023-08-25T04:42:50.326902+0000","last_active":"2023-08-25T04:43:34.762343+0000","last_peered":"2023-08-25T04:43:34.762343+0000","last_clean":"2023-08-25T04:43:34.762343+0000","last_became_active":"2023-08-25T04:42:50.326552+0000","last_became_peered":"2023-08-25T04:42:50.326552+0000","last_unstale":"2023-08-25T04:43:34.762343+0000","last_undegraded":"2023-08-25T04:43:34.762343+0000","last_fullsized":"2023-08-25T04:43:34.762343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:57:35.343381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766096+0000","last_change":"2023-08-25T04:42:50.327715+0000","last_active":"2023-08-25T04:43:34.766096+0000","last_peered":"2023-08-25T04:43:34.766096+0000","last_clean":"2023-08-25T04:43:34.766096+0000","last_became_active":"2023-08-25T04:42:50.327385+0000","last_became_peered":"2023-08-25T04:42:50.327385+0000","last_unstale":"2023-08-25T04:43:34.766096+0000","last_undegraded":"2023-08-25T04:43:34.766096+0000","last_fullsized":"2023-08-25T04:43:34.766096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:53:26.027535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764535+0000","last_change":"2023-08-25T04:42:50.330004+0000","last_active":"2023-08-25T04:43:34.764535+0000","last_peered":"2023-08-25T04:43:34.764535+0000","last_clean":"2023-08-25T04:43:34.764535+0000","last_became_active":"2023-08-25T04:42:50.329629+0000","last_became_peered":"2023-08-25T04:42:50.329629+0000","last_unstale":"2023-08-25T04:43:34.764535+0000","last_undegraded":"2023-08-25T04:43:34.764535+0000","last_fullsized":"2023-08-25T04:43:34.764535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:00.744928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922718+0000","last_change":"2023-08-25T04:42:50.335345+0000","last_active":"2023-08-25T04:43:34.922718+0000","last_peered":"2023-08-25T04:43:34.922718+0000","last_clean":"2023-08-25T04:43:34.922718+0000","last_became_active":"2023-08-25T04:42:50.335031+0000","last_became_peered":"2023-08-25T04:42:50.335031+0000","last_unstale":"2023-08-25T04:43:34.922718+0000","last_undegraded":"2023-08-25T04:43:34.922718+0000","last_fullsized":"2023-08-25T04:43:34.922718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:47:46.522466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765008+0000","last_change":"2023-08-25T04:42:50.334891+0000","last_active":"2023-08-25T04:43:34.765008+0000","last_peered":"2023-08-25T04:43:34.765008+0000","last_clean":"2023-08-25T04:43:34.765008+0000","last_became_active":"2023-08-25T04:42:50.334569+0000","last_became_peered":"2023-08-25T04:42:50.334569+0000","last_unstale":"2023-08-25T04:43:34.765008+0000","last_undegraded":"2023-08-25T04:43:34.765008+0000","last_fullsized":"2023-08-25T04:43:34.765008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:08:38.855500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099337+0000","last_change":"2023-08-25T04:42:50.335209+0000","last_active":"2023-08-25T04:43:35.099337+0000","last_peered":"2023-08-25T04:43:35.099337+0000","last_clean":"2023-08-25T04:43:35.099337+0000","last_became_active":"2023-08-25T04:42:50.334904+0000","last_became_peered":"2023-08-25T04:42:50.334904+0000","last_unstale":"2023-08-25T04:43:35.099337+0000","last_undegraded":"2023-08-25T04:43:35.099337+0000","last_fullsized":"2023-08-25T04:43:35.099337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:36.964244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765117+0000","last_change":"2023-08-25T04:42:50.336028+0000","last_active":"2023-08-25T04:43:34.765117+0000","last_peered":"2023-08-25T04:43:34.765117+0000","last_clean":"2023-08-25T04:43:34.765117+0000","last_became_active":"2023-08-25T04:42:50.330836+0000","last_became_peered":"2023-08-25T04:42:50.330836+0000","last_unstale":"2023-08-25T04:43:34.765117+0000","last_undegraded":"2023-08-25T04:43:34.765117+0000","last_fullsized":"2023-08-25T04:43:34.765117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:00.051214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923189+0000","last_change":"2023-08-25T04:42:50.328583+0000","last_active":"2023-08-25T04:43:34.923189+0000","last_peered":"2023-08-25T04:43:34.923189+0000","last_clean":"2023-08-25T04:43:34.923189+0000","last_became_active":"2023-08-25T04:42:50.328260+0000","last_became_peered":"2023-08-25T04:42:50.328260+0000","last_unstale":"2023-08-25T04:43:34.923189+0000","last_undegraded":"2023-08-25T04:43:34.923189+0000","last_fullsized":"2023-08-25T04:43:34.923189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:24.087589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.762199+0000","last_change":"2023-08-25T04:42:50.329495+0000","last_active":"2023-08-25T04:43:34.762199+0000","last_peered":"2023-08-25T04:43:34.762199+0000","last_clean":"2023-08-25T04:43:34.762199+0000","last_became_active":"2023-08-25T04:42:50.329094+0000","last_became_peered":"2023-08-25T04:42:50.329094+0000","last_unstale":"2023-08-25T04:43:34.762199+0000","last_undegraded":"2023-08-25T04:43:34.762199+0000","last_fullsized":"2023-08-25T04:43:34.762199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:23.029577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099138+0000","last_change":"2023-08-25T04:42:50.336495+0000","last_active":"2023-08-25T04:43:35.099138+0000","last_peered":"2023-08-25T04:43:35.099138+0000","last_clean":"2023-08-25T04:43:35.099138+0000","last_became_active":"2023-08-25T04:42:50.336155+0000","last_became_peered":"2023-08-25T04:42:50.336155+0000","last_unstale":"2023-08-25T04:43:35.099138+0000","last_undegraded":"2023-08-25T04:43:35.099138+0000","last_fullsized":"2023-08-25T04:43:35.099138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:46:53.794589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764677+0000","last_change":"2023-08-25T04:42:50.327218+0000","last_active":"2023-08-25T04:43:34.764677+0000","last_peered":"2023-08-25T04:43:34.764677+0000","last_clean":"2023-08-25T04:43:34.764677+0000","last_became_active":"2023-08-25T04:42:50.326881+0000","last_became_peered":"2023-08-25T04:42:50.326881+0000","last_unstale":"2023-08-25T04:43:34.764677+0000","last_undegraded":"2023-08-25T04:43:34.764677+0000","last_fullsized":"2023-08-25T04:43:34.764677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:07.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764876+0000","last_change":"2023-08-25T04:42:50.334361+0000","last_active":"2023-08-25T04:43:34.764876+0000","last_peered":"2023-08-25T04:43:34.764876+0000","last_clean":"2023-08-25T04:43:34.764876+0000","last_became_active":"2023-08-25T04:42:50.334069+0000","last_became_peered":"2023-08-25T04:42:50.334069+0000","last_unstale":"2023-08-25T04:43:34.764876+0000","last_undegraded":"2023-08-25T04:43:34.764876+0000","last_fullsized":"2023-08-25T04:43:34.764876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:15:49.317438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098418+0000","last_change":"2023-08-25T04:42:50.337152+0000","last_active":"2023-08-25T04:43:35.098418+0000","last_peered":"2023-08-25T04:43:35.098418+0000","last_clean":"2023-08-25T04:43:35.098418+0000","last_became_active":"2023-08-25T04:42:50.336855+0000","last_became_peered":"2023-08-25T04:42:50.336855+0000","last_unstale":"2023-08-25T04:43:35.098418+0000","last_undegraded":"2023-08-25T04:43:35.098418+0000","last_fullsized":"2023-08-25T04:43:35.098418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:53.277068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765316+0000","last_change":"2023-08-25T04:42:50.327308+0000","last_active":"2023-08-25T04:43:34.765316+0000","last_peered":"2023-08-25T04:43:34.765316+0000","last_clean":"2023-08-25T04:43:34.765316+0000","last_became_active":"2023-08-25T04:42:50.327000+0000","last_became_peered":"2023-08-25T04:42:50.327000+0000","last_unstale":"2023-08-25T04:43:34.765316+0000","last_undegraded":"2023-08-25T04:43:34.765316+0000","last_fullsized":"2023-08-25T04:43:34.765316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:04.508809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761810+0000","last_change":"2023-08-25T04:42:50.330938+0000","last_active":"2023-08-25T04:43:34.761810+0000","last_peered":"2023-08-25T04:43:34.761810+0000","last_clean":"2023-08-25T04:43:34.761810+0000","last_became_active":"2023-08-25T04:42:50.330345+0000","last_became_peered":"2023-08-25T04:42:50.330345+0000","last_unstale":"2023-08-25T04:43:34.761810+0000","last_undegraded":"2023-08-25T04:43:34.761810+0000","last_fullsized":"2023-08-25T04:43:34.761810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:10:02.382799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770010+0000","last_change":"2023-08-25T04:42:50.333716+0000","last_active":"2023-08-25T04:43:34.770010+0000","last_peered":"2023-08-25T04:43:34.770010+0000","last_clean":"2023-08-25T04:43:34.770010+0000","last_became_active":"2023-08-25T04:42:50.332993+0000","last_became_peered":"2023-08-25T04:42:50.332993+0000","last_unstale":"2023-08-25T04:43:34.770010+0000","last_undegraded":"2023-08-25T04:43:34.770010+0000","last_fullsized":"2023-08-25T04:43:34.770010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:00:34.306505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769760+0000","last_change":"2023-08-25T04:42:54.358089+0000","last_active":"2023-08-25T04:43:34.769760+0000","last_peered":"2023-08-25T04:43:34.769760+0000","last_clean":"2023-08-25T04:43:34.769760+0000","last_became_active":"2023-08-25T04:42:54.357726+0000","last_became_peered":"2023-08-25T04:42:54.357726+0000","last_unstale":"2023-08-25T04:43:34.769760+0000","last_undegraded":"2023-08-25T04:43:34.769760+0000","last_fullsized":"2023-08-25T04:43:34.769760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:23.911843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761620+0000","last_change":"2023-08-25T04:42:49.330320+0000","last_active":"2023-08-25T04:43:34.761620+0000","last_peered":"2023-08-25T04:43:34.761620+0000","last_clean":"2023-08-25T04:43:34.761620+0000","last_became_active":"2023-08-25T04:42:49.330020+0000","last_became_peered":"2023-08-25T04:42:49.330020+0000","last_unstale":"2023-08-25T04:43:34.761620+0000","last_undegraded":"2023-08-25T04:43:34.761620+0000","last_fullsized":"2023-08-25T04:43:34.761620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:12:30.639568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765547+0000","last_change":"2023-08-25T04:42:50.337487+0000","last_active":"2023-08-25T04:43:34.765547+0000","last_peered":"2023-08-25T04:43:34.765547+0000","last_clean":"2023-08-25T04:43:34.765547+0000","last_became_active":"2023-08-25T04:42:50.337137+0000","last_became_peered":"2023-08-25T04:42:50.337137+0000","last_unstale":"2023-08-25T04:43:34.765547+0000","last_undegraded":"2023-08-25T04:43:34.765547+0000","last_fullsized":"2023-08-25T04:43:34.765547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:24.659622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100513+0000","last_change":"2023-08-25T04:43:35.100513+0000","last_active":"2023-08-25T04:43:35.100513+0000","last_peered":"2023-08-25T04:43:35.100513+0000","last_clean":"2023-08-25T04:43:35.100513+0000","last_became_active":"2023-08-25T04:43:35.100067+0000","last_became_peered":"2023-08-25T04:43:35.100067+0000","last_unstale":"2023-08-25T04:43:35.100513+0000","last_undegraded":"2023-08-25T04:43:35.100513+0000","last_fullsized":"2023-08-25T04:43:35.100513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:23:03.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765693+0000","last_change":"2023-08-25T04:42:49.333053+0000","last_active":"2023-08-25T04:43:34.765693+0000","last_peered":"2023-08-25T04:43:34.765693+0000","last_clean":"2023-08-25T04:43:34.765693+0000","last_became_active":"2023-08-25T04:42:49.332740+0000","last_became_peered":"2023-08-25T04:42:49.332740+0000","last_unstale":"2023-08-25T04:43:34.765693+0000","last_undegraded":"2023-08-25T04:43:34.765693+0000","last_fullsized":"2023-08-25T04:43:34.765693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:38:39.362078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765706+0000","last_change":"2023-08-25T04:42:50.330084+0000","last_active":"2023-08-25T04:43:34.765706+0000","last_peered":"2023-08-25T04:43:34.765706+0000","last_clean":"2023-08-25T04:43:34.765706+0000","last_became_active":"2023-08-25T04:42:50.329681+0000","last_became_peered":"2023-08-25T04:42:50.329681+0000","last_unstale":"2023-08-25T04:43:34.765706+0000","last_undegraded":"2023-08-25T04:43:34.765706+0000","last_fullsized":"2023-08-25T04:43:34.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:25.041276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761550+0000","last_change":"2023-08-25T04:42:54.358546+0000","last_active":"2023-08-25T04:43:34.761550+0000","last_peered":"2023-08-25T04:43:34.761550+0000","last_clean":"2023-08-25T04:43:34.761550+0000","last_became_active":"2023-08-25T04:42:54.358129+0000","last_became_peered":"2023-08-25T04:42:54.358129+0000","last_unstale":"2023-08-25T04:43:34.761550+0000","last_undegraded":"2023-08-25T04:43:34.761550+0000","last_fullsized":"2023-08-25T04:43:34.761550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:25.771337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765423+0000","last_change":"2023-08-25T04:42:49.328530+0000","last_active":"2023-08-25T04:43:34.765423+0000","last_peered":"2023-08-25T04:43:34.765423+0000","last_clean":"2023-08-25T04:43:34.765423+0000","last_became_active":"2023-08-25T04:42:49.328211+0000","last_became_peered":"2023-08-25T04:42:49.328211+0000","last_unstale":"2023-08-25T04:43:34.765423+0000","last_undegraded":"2023-08-25T04:43:34.765423+0000","last_fullsized":"2023-08-25T04:43:34.765423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:27.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098820+0000","last_change":"2023-08-25T04:42:50.328736+0000","last_active":"2023-08-25T04:43:35.098820+0000","last_peered":"2023-08-25T04:43:35.098820+0000","last_clean":"2023-08-25T04:43:35.098820+0000","last_became_active":"2023-08-25T04:42:50.328412+0000","last_became_peered":"2023-08-25T04:42:50.328412+0000","last_unstale":"2023-08-25T04:43:35.098820+0000","last_undegraded":"2023-08-25T04:43:35.098820+0000","last_fullsized":"2023-08-25T04:43:35.098820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:20.999420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761408+0000","last_change":"2023-08-25T04:42:54.352836+0000","last_active":"2023-08-25T04:43:34.761408+0000","last_peered":"2023-08-25T04:43:34.761408+0000","last_clean":"2023-08-25T04:43:34.761408+0000","last_became_active":"2023-08-25T04:42:54.352437+0000","last_became_peered":"2023-08-25T04:42:54.352437+0000","last_unstale":"2023-08-25T04:43:34.761408+0000","last_undegraded":"2023-08-25T04:43:34.761408+0000","last_fullsized":"2023-08-25T04:43:34.761408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:14.562518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764854+0000","last_change":"2023-08-25T04:42:49.322218+0000","last_active":"2023-08-25T04:43:34.764854+0000","last_peered":"2023-08-25T04:43:34.764854+0000","last_clean":"2023-08-25T04:43:34.764854+0000","last_became_active":"2023-08-25T04:42:49.321905+0000","last_became_peered":"2023-08-25T04:42:49.321905+0000","last_unstale":"2023-08-25T04:43:34.764854+0000","last_undegraded":"2023-08-25T04:43:34.764854+0000","last_fullsized":"2023-08-25T04:43:34.764854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:27.603878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099267+0000","last_change":"2023-08-25T04:42:50.332988+0000","last_active":"2023-08-25T04:43:35.099267+0000","last_peered":"2023-08-25T04:43:35.099267+0000","last_clean":"2023-08-25T04:43:35.099267+0000","last_became_active":"2023-08-25T04:42:50.332483+0000","last_became_peered":"2023-08-25T04:42:50.332483+0000","last_unstale":"2023-08-25T04:43:35.099267+0000","last_undegraded":"2023-08-25T04:43:35.099267+0000","last_fullsized":"2023-08-25T04:43:35.099267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:28.457166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769245+0000","last_change":"2023-08-25T04:42:54.357347+0000","last_active":"2023-08-25T04:43:34.769245+0000","last_peered":"2023-08-25T04:43:34.769245+0000","last_clean":"2023-08-25T04:43:34.769245+0000","last_became_active":"2023-08-25T04:42:54.356898+0000","last_became_peered":"2023-08-25T04:42:54.356898+0000","last_unstale":"2023-08-25T04:43:34.769245+0000","last_undegraded":"2023-08-25T04:43:34.769245+0000","last_fullsized":"2023-08-25T04:43:34.769245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:47.099556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":127,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.695688+0000","last_change":"2023-08-25T04:42:49.325245+0000","last_active":"2023-08-25T04:44:46.695688+0000","last_peered":"2023-08-25T04:44:46.695688+0000","last_clean":"2023-08-25T04:44:46.695688+0000","last_became_active":"2023-08-25T04:42:49.324936+0000","last_became_peered":"2023-08-25T04:42:49.324936+0000","last_unstale":"2023-08-25T04:44:46.695688+0000","last_undegraded":"2023-08-25T04:44:46.695688+0000","last_fullsized":"2023-08-25T04:44:46.695688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:02.879296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765116+0000","last_change":"2023-08-25T04:42:50.333726+0000","last_active":"2023-08-25T04:43:34.765116+0000","last_peered":"2023-08-25T04:43:34.765116+0000","last_clean":"2023-08-25T04:43:34.765116+0000","last_became_active":"2023-08-25T04:42:50.333425+0000","last_became_peered":"2023-08-25T04:42:50.333425+0000","last_unstale":"2023-08-25T04:43:34.765116+0000","last_undegraded":"2023-08-25T04:43:34.765116+0000","last_fullsized":"2023-08-25T04:43:34.765116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:18.764324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769440+0000","last_change":"2023-08-25T04:42:54.362817+0000","last_active":"2023-08-25T04:43:34.769440+0000","last_peered":"2023-08-25T04:43:34.769440+0000","last_clean":"2023-08-25T04:43:34.769440+0000","last_became_active":"2023-08-25T04:42:54.362373+0000","last_became_peered":"2023-08-25T04:42:54.362373+0000","last_unstale":"2023-08-25T04:43:34.769440+0000","last_undegraded":"2023-08-25T04:43:34.769440+0000","last_fullsized":"2023-08-25T04:43:34.769440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:38.395114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764497+0000","last_change":"2023-08-25T04:42:49.324422+0000","last_active":"2023-08-25T04:43:34.764497+0000","last_peered":"2023-08-25T04:43:34.764497+0000","last_clean":"2023-08-25T04:43:34.764497+0000","last_became_active":"2023-08-25T04:42:49.324124+0000","last_became_peered":"2023-08-25T04:42:49.324124+0000","last_unstale":"2023-08-25T04:43:34.764497+0000","last_undegraded":"2023-08-25T04:43:34.764497+0000","last_fullsized":"2023-08-25T04:43:34.764497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:29:44.193254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099118+0000","last_change":"2023-08-25T04:42:50.332900+0000","last_active":"2023-08-25T04:43:35.099118+0000","last_peered":"2023-08-25T04:43:35.099118+0000","last_clean":"2023-08-25T04:43:35.099118+0000","last_became_active":"2023-08-25T04:42:50.332540+0000","last_became_peered":"2023-08-25T04:42:50.332540+0000","last_unstale":"2023-08-25T04:43:35.099118+0000","last_undegraded":"2023-08-25T04:43:35.099118+0000","last_fullsized":"2023-08-25T04:43:35.099118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:43.013041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765157+0000","last_change":"2023-08-25T04:42:54.357754+0000","last_active":"2023-08-25T04:43:34.765157+0000","last_peered":"2023-08-25T04:43:34.765157+0000","last_clean":"2023-08-25T04:43:34.765157+0000","last_became_active":"2023-08-25T04:42:54.357318+0000","last_became_peered":"2023-08-25T04:42:54.357318+0000","last_unstale":"2023-08-25T04:43:34.765157+0000","last_undegraded":"2023-08-25T04:43:34.765157+0000","last_fullsized":"2023-08-25T04:43:34.765157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:14.536121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098445+0000","last_change":"2023-08-25T04:42:49.320372+0000","last_active":"2023-08-25T04:43:35.098445+0000","last_peered":"2023-08-25T04:43:35.098445+0000","last_clean":"2023-08-25T04:43:35.098445+0000","last_became_active":"2023-08-25T04:42:49.319977+0000","last_became_peered":"2023-08-25T04:42:49.319977+0000","last_unstale":"2023-08-25T04:43:35.098445+0000","last_undegraded":"2023-08-25T04:43:35.098445+0000","last_fullsized":"2023-08-25T04:43:35.098445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:45.583226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764897+0000","last_change":"2023-08-25T04:42:50.327564+0000","last_active":"2023-08-25T04:43:34.764897+0000","last_peered":"2023-08-25T04:43:34.764897+0000","last_clean":"2023-08-25T04:43:34.764897+0000","last_became_active":"2023-08-25T04:42:50.327249+0000","last_became_peered":"2023-08-25T04:42:50.327249+0000","last_unstale":"2023-08-25T04:43:34.764897+0000","last_undegraded":"2023-08-25T04:43:34.764897+0000","last_fullsized":"2023-08-25T04:43:34.764897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:04.925252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764614+0000","last_change":"2023-08-25T04:42:54.354150+0000","last_active":"2023-08-25T04:43:34.764614+0000","last_peered":"2023-08-25T04:43:34.764614+0000","last_clean":"2023-08-25T04:43:34.764614+0000","last_became_active":"2023-08-25T04:42:54.353768+0000","last_became_peered":"2023-08-25T04:42:54.353768+0000","last_unstale":"2023-08-25T04:43:34.764614+0000","last_undegraded":"2023-08-25T04:43:34.764614+0000","last_fullsized":"2023-08-25T04:43:34.764614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:24.431309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098697+0000","last_change":"2023-08-25T04:42:49.325079+0000","last_active":"2023-08-25T04:43:35.098697+0000","last_peered":"2023-08-25T04:43:35.098697+0000","last_clean":"2023-08-25T04:43:35.098697+0000","last_became_active":"2023-08-25T04:42:49.324772+0000","last_became_peered":"2023-08-25T04:42:49.324772+0000","last_unstale":"2023-08-25T04:43:35.098697+0000","last_undegraded":"2023-08-25T04:43:35.098697+0000","last_fullsized":"2023-08-25T04:43:35.098697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:49:26.668766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761101+0000","last_change":"2023-08-25T04:42:50.325223+0000","last_active":"2023-08-25T04:43:34.761101+0000","last_peered":"2023-08-25T04:43:34.761101+0000","last_clean":"2023-08-25T04:43:34.761101+0000","last_became_active":"2023-08-25T04:42:50.324916+0000","last_became_peered":"2023-08-25T04:42:50.324916+0000","last_unstale":"2023-08-25T04:43:34.761101+0000","last_undegraded":"2023-08-25T04:43:34.761101+0000","last_fullsized":"2023-08-25T04:43:34.761101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:51.398185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765545+0000","last_change":"2023-08-25T04:42:54.358904+0000","last_active":"2023-08-25T04:43:34.765545+0000","last_peered":"2023-08-25T04:43:34.765545+0000","last_clean":"2023-08-25T04:43:34.765545+0000","last_became_active":"2023-08-25T04:42:54.358401+0000","last_became_peered":"2023-08-25T04:42:54.358401+0000","last_unstale":"2023-08-25T04:43:34.765545+0000","last_undegraded":"2023-08-25T04:43:34.765545+0000","last_fullsized":"2023-08-25T04:43:34.765545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:28.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:47.696571+0000","last_change":"2023-08-25T04:42:49.331367+0000","last_active":"2023-08-25T04:44:47.696571+0000","last_peered":"2023-08-25T04:44:47.696571+0000","last_clean":"2023-08-25T04:44:47.696571+0000","last_became_active":"2023-08-25T04:42:49.331110+0000","last_became_peered":"2023-08-25T04:42:49.331110+0000","last_unstale":"2023-08-25T04:44:47.696571+0000","last_undegraded":"2023-08-25T04:44:47.696571+0000","last_fullsized":"2023-08-25T04:44:47.696571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:44.974722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099154+0000","last_change":"2023-08-25T04:42:50.329541+0000","last_active":"2023-08-25T04:43:35.099154+0000","last_peered":"2023-08-25T04:43:35.099154+0000","last_clean":"2023-08-25T04:43:35.099154+0000","last_became_active":"2023-08-25T04:42:50.329242+0000","last_became_peered":"2023-08-25T04:42:50.329242+0000","last_unstale":"2023-08-25T04:43:35.099154+0000","last_undegraded":"2023-08-25T04:43:35.099154+0000","last_fullsized":"2023-08-25T04:43:35.099154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:07.872117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761225+0000","last_change":"2023-08-25T04:42:54.358533+0000","last_active":"2023-08-25T04:43:34.761225+0000","last_peered":"2023-08-25T04:43:34.761225+0000","last_clean":"2023-08-25T04:43:34.761225+0000","last_became_active":"2023-08-25T04:42:54.358166+0000","last_became_peered":"2023-08-25T04:42:54.358166+0000","last_unstale":"2023-08-25T04:43:34.761225+0000","last_undegraded":"2023-08-25T04:43:34.761225+0000","last_fullsized":"2023-08-25T04:43:34.761225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:18.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923185+0000","last_change":"2023-08-25T04:42:49.325341+0000","last_active":"2023-08-25T04:43:34.923185+0000","last_peered":"2023-08-25T04:43:34.923185+0000","last_clean":"2023-08-25T04:43:34.923185+0000","last_became_active":"2023-08-25T04:42:49.325050+0000","last_became_peered":"2023-08-25T04:42:49.325050+0000","last_unstale":"2023-08-25T04:43:34.923185+0000","last_undegraded":"2023-08-25T04:43:34.923185+0000","last_fullsized":"2023-08-25T04:43:34.923185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:14.764732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766754+0000","last_change":"2023-08-25T04:42:50.336895+0000","last_active":"2023-08-25T04:43:34.766754+0000","last_peered":"2023-08-25T04:43:34.766754+0000","last_clean":"2023-08-25T04:43:34.766754+0000","last_became_active":"2023-08-25T04:42:50.336594+0000","last_became_peered":"2023-08-25T04:42:50.336594+0000","last_unstale":"2023-08-25T04:43:34.766754+0000","last_undegraded":"2023-08-25T04:43:34.766754+0000","last_fullsized":"2023-08-25T04:43:34.766754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:01.393459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100106+0000","last_change":"2023-08-25T04:43:35.100106+0000","last_active":"2023-08-25T04:43:35.100106+0000","last_peered":"2023-08-25T04:43:35.100106+0000","last_clean":"2023-08-25T04:43:35.100106+0000","last_became_active":"2023-08-25T04:43:35.099546+0000","last_became_peered":"2023-08-25T04:43:35.099546+0000","last_unstale":"2023-08-25T04:43:35.100106+0000","last_undegraded":"2023-08-25T04:43:35.100106+0000","last_fullsized":"2023-08-25T04:43:35.100106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:13:25.055771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099268+0000","last_change":"2023-08-25T04:42:49.325586+0000","last_active":"2023-08-25T04:43:35.099268+0000","last_peered":"2023-08-25T04:43:35.099268+0000","last_clean":"2023-08-25T04:43:35.099268+0000","last_became_active":"2023-08-25T04:42:49.324486+0000","last_became_peered":"2023-08-25T04:42:49.324486+0000","last_unstale":"2023-08-25T04:43:35.099268+0000","last_undegraded":"2023-08-25T04:43:35.099268+0000","last_fullsized":"2023-08-25T04:43:35.099268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:54.567632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098918+0000","last_change":"2023-08-25T04:42:50.330314+0000","last_active":"2023-08-25T04:43:35.098918+0000","last_peered":"2023-08-25T04:43:35.098918+0000","last_clean":"2023-08-25T04:43:35.098918+0000","last_became_active":"2023-08-25T04:42:50.329924+0000","last_became_peered":"2023-08-25T04:42:50.329924+0000","last_unstale":"2023-08-25T04:43:35.098918+0000","last_undegraded":"2023-08-25T04:43:35.098918+0000","last_fullsized":"2023-08-25T04:43:35.098918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:47.455399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761107+0000","last_change":"2023-08-25T04:42:54.355617+0000","last_active":"2023-08-25T04:43:34.761107+0000","last_peered":"2023-08-25T04:43:34.761107+0000","last_clean":"2023-08-25T04:43:34.761107+0000","last_became_active":"2023-08-25T04:42:54.355161+0000","last_became_peered":"2023-08-25T04:42:54.355161+0000","last_unstale":"2023-08-25T04:43:34.761107+0000","last_undegraded":"2023-08-25T04:43:34.761107+0000","last_fullsized":"2023-08-25T04:43:34.761107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:27:18.610077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:48.697368+0000","last_change":"2023-08-25T04:42:49.325042+0000","last_active":"2023-08-25T04:44:48.697368+0000","last_peered":"2023-08-25T04:44:48.697368+0000","last_clean":"2023-08-25T04:44:48.697368+0000","last_became_active":"2023-08-25T04:42:49.324787+0000","last_became_peered":"2023-08-25T04:42:49.324787+0000","last_unstale":"2023-08-25T04:44:48.697368+0000","last_undegraded":"2023-08-25T04:44:48.697368+0000","last_fullsized":"2023-08-25T04:44:48.697368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:00.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":188,"num_read_kb":98,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765074+0000","last_change":"2023-08-25T04:42:50.336627+0000","last_active":"2023-08-25T04:43:34.765074+0000","last_peered":"2023-08-25T04:43:34.765074+0000","last_clean":"2023-08-25T04:43:34.765074+0000","last_became_active":"2023-08-25T04:42:50.336364+0000","last_became_peered":"2023-08-25T04:42:50.336364+0000","last_unstale":"2023-08-25T04:43:34.765074+0000","last_undegraded":"2023-08-25T04:43:34.765074+0000","last_fullsized":"2023-08-25T04:43:34.765074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:07.231855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099639+0000","last_change":"2023-08-25T04:42:54.350824+0000","last_active":"2023-08-25T04:43:35.099639+0000","last_peered":"2023-08-25T04:43:35.099639+0000","last_clean":"2023-08-25T04:43:35.099639+0000","last_became_active":"2023-08-25T04:42:54.350328+0000","last_became_peered":"2023-08-25T04:42:54.350328+0000","last_unstale":"2023-08-25T04:43:35.099639+0000","last_undegraded":"2023-08-25T04:43:35.099639+0000","last_fullsized":"2023-08-25T04:43:35.099639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:46:34.120949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769705+0000","last_change":"2023-08-25T04:42:49.327923+0000","last_active":"2023-08-25T04:43:34.769705+0000","last_peered":"2023-08-25T04:43:34.769705+0000","last_clean":"2023-08-25T04:43:34.769705+0000","last_became_active":"2023-08-25T04:42:49.327195+0000","last_became_peered":"2023-08-25T04:42:49.327195+0000","last_unstale":"2023-08-25T04:43:34.769705+0000","last_undegraded":"2023-08-25T04:43:34.769705+0000","last_fullsized":"2023-08-25T04:43:34.769705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:00.803708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098704+0000","last_change":"2023-08-25T04:42:50.325574+0000","last_active":"2023-08-25T04:43:35.098704+0000","last_peered":"2023-08-25T04:43:35.098704+0000","last_clean":"2023-08-25T04:43:35.098704+0000","last_became_active":"2023-08-25T04:42:50.325244+0000","last_became_peered":"2023-08-25T04:42:50.325244+0000","last_unstale":"2023-08-25T04:43:35.098704+0000","last_undegraded":"2023-08-25T04:43:35.098704+0000","last_fullsized":"2023-08-25T04:43:35.098704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:19:57.215688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.767465+0000","last_change":"2023-08-25T04:42:54.357622+0000","last_active":"2023-08-25T04:43:34.767465+0000","last_peered":"2023-08-25T04:43:34.767465+0000","last_clean":"2023-08-25T04:43:34.767465+0000","last_became_active":"2023-08-25T04:42:54.357151+0000","last_became_peered":"2023-08-25T04:42:54.357151+0000","last_unstale":"2023-08-25T04:43:34.767465+0000","last_undegraded":"2023-08-25T04:43:34.767465+0000","last_fullsized":"2023-08-25T04:43:34.767465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:22.379000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098949+0000","last_change":"2023-08-25T04:42:49.328334+0000","last_active":"2023-08-25T04:43:35.098949+0000","last_peered":"2023-08-25T04:43:35.098949+0000","last_clean":"2023-08-25T04:43:35.098949+0000","last_became_active":"2023-08-25T04:42:49.328037+0000","last_became_peered":"2023-08-25T04:42:49.328037+0000","last_unstale":"2023-08-25T04:43:35.098949+0000","last_undegraded":"2023-08-25T04:43:35.098949+0000","last_fullsized":"2023-08-25T04:43:35.098949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:38.602930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923061+0000","last_change":"2023-08-25T04:42:50.326904+0000","last_active":"2023-08-25T04:43:34.923061+0000","last_peered":"2023-08-25T04:43:34.923061+0000","last_clean":"2023-08-25T04:43:34.923061+0000","last_became_active":"2023-08-25T04:42:50.325941+0000","last_became_peered":"2023-08-25T04:42:50.325941+0000","last_unstale":"2023-08-25T04:43:34.923061+0000","last_undegraded":"2023-08-25T04:43:34.923061+0000","last_fullsized":"2023-08-25T04:43:34.923061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:57.719232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761321+0000","last_change":"2023-08-25T04:42:54.353877+0000","last_active":"2023-08-25T04:43:34.761321+0000","last_peered":"2023-08-25T04:43:34.761321+0000","last_clean":"2023-08-25T04:43:34.761321+0000","last_became_active":"2023-08-25T04:42:54.353502+0000","last_became_peered":"2023-08-25T04:42:54.353502+0000","last_unstale":"2023-08-25T04:43:34.761321+0000","last_undegraded":"2023-08-25T04:43:34.761321+0000","last_fullsized":"2023-08-25T04:43:34.761321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:45.122188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.718684+0000","last_change":"2023-08-25T04:42:49.327356+0000","last_active":"2023-08-25T04:44:46.718684+0000","last_peered":"2023-08-25T04:44:46.718684+0000","last_clean":"2023-08-25T04:44:46.718684+0000","last_became_active":"2023-08-25T04:42:49.327048+0000","last_became_peered":"2023-08-25T04:42:49.327048+0000","last_unstale":"2023-08-25T04:44:46.718684+0000","last_undegraded":"2023-08-25T04:44:46.718684+0000","last_fullsized":"2023-08-25T04:44:46.718684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:03:39.958808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922765+0000","last_change":"2023-08-25T04:42:50.333698+0000","last_active":"2023-08-25T04:43:34.922765+0000","last_peered":"2023-08-25T04:43:34.922765+0000","last_clean":"2023-08-25T04:43:34.922765+0000","last_became_active":"2023-08-25T04:42:50.333395+0000","last_became_peered":"2023-08-25T04:42:50.333395+0000","last_unstale":"2023-08-25T04:43:34.922765+0000","last_undegraded":"2023-08-25T04:43:34.922765+0000","last_fullsized":"2023-08-25T04:43:34.922765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:10:22.482680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099199+0000","last_change":"2023-08-25T04:42:54.355502+0000","last_active":"2023-08-25T04:43:35.099199+0000","last_peered":"2023-08-25T04:43:35.099199+0000","last_clean":"2023-08-25T04:43:35.099199+0000","last_became_active":"2023-08-25T04:42:54.355118+0000","last_became_peered":"2023-08-25T04:42:54.355118+0000","last_unstale":"2023-08-25T04:43:35.099199+0000","last_undegraded":"2023-08-25T04:43:35.099199+0000","last_fullsized":"2023-08-25T04:43:35.099199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:42:46.918540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765654+0000","last_change":"2023-08-25T04:42:49.325599+0000","last_active":"2023-08-25T04:43:34.765654+0000","last_peered":"2023-08-25T04:43:34.765654+0000","last_clean":"2023-08-25T04:43:34.765654+0000","last_became_active":"2023-08-25T04:42:49.324845+0000","last_became_peered":"2023-08-25T04:42:49.324845+0000","last_unstale":"2023-08-25T04:43:34.765654+0000","last_undegraded":"2023-08-25T04:43:34.765654+0000","last_fullsized":"2023-08-25T04:43:34.765654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:28.897000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.337951+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.337667+0000","last_became_peered":"2023-08-25T04:42:50.337667+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:37:02.027948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098592+0000","last_change":"2023-08-25T04:42:54.349601+0000","last_active":"2023-08-25T04:43:35.098592+0000","last_peered":"2023-08-25T04:43:35.098592+0000","last_clean":"2023-08-25T04:43:35.098592+0000","last_became_active":"2023-08-25T04:42:54.349140+0000","last_became_peered":"2023-08-25T04:42:54.349140+0000","last_unstale":"2023-08-25T04:43:35.098592+0000","last_undegraded":"2023-08-25T04:43:35.098592+0000","last_fullsized":"2023-08-25T04:43:35.098592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:19:33.883508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764534+0000","last_change":"2023-08-25T04:42:49.317294+0000","last_active":"2023-08-25T04:43:34.764534+0000","last_peered":"2023-08-25T04:43:34.764534+0000","last_clean":"2023-08-25T04:43:34.764534+0000","last_became_active":"2023-08-25T04:42:49.316984+0000","last_became_peered":"2023-08-25T04:42:49.316984+0000","last_unstale":"2023-08-25T04:43:34.764534+0000","last_undegraded":"2023-08-25T04:43:34.764534+0000","last_fullsized":"2023-08-25T04:43:34.764534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:04:26.603958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766834+0000","last_change":"2023-08-25T04:42:50.335084+0000","last_active":"2023-08-25T04:43:34.766834+0000","last_peered":"2023-08-25T04:43:34.766834+0000","last_clean":"2023-08-25T04:43:34.766834+0000","last_became_active":"2023-08-25T04:42:50.334742+0000","last_became_peered":"2023-08-25T04:42:50.334742+0000","last_unstale":"2023-08-25T04:43:34.766834+0000","last_undegraded":"2023-08-25T04:43:34.766834+0000","last_fullsized":"2023-08-25T04:43:34.766834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:35:18.420398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764942+0000","last_change":"2023-08-25T04:42:54.354188+0000","last_active":"2023-08-25T04:43:34.764942+0000","last_peered":"2023-08-25T04:43:34.764942+0000","last_clean":"2023-08-25T04:43:34.764942+0000","last_became_active":"2023-08-25T04:42:54.353707+0000","last_became_peered":"2023-08-25T04:42:54.353707+0000","last_unstale":"2023-08-25T04:43:34.764942+0000","last_undegraded":"2023-08-25T04:43:34.764942+0000","last_fullsized":"2023-08-25T04:43:34.764942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:31.579981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761767+0000","last_change":"2023-08-25T04:42:49.326256+0000","last_active":"2023-08-25T04:43:34.761767+0000","last_peered":"2023-08-25T04:43:34.761767+0000","last_clean":"2023-08-25T04:43:34.761767+0000","last_became_active":"2023-08-25T04:42:49.325884+0000","last_became_peered":"2023-08-25T04:42:49.325884+0000","last_unstale":"2023-08-25T04:43:34.761767+0000","last_undegraded":"2023-08-25T04:43:34.761767+0000","last_fullsized":"2023-08-25T04:43:34.761767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:59:32.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764777+0000","last_change":"2023-08-25T04:42:50.330037+0000","last_active":"2023-08-25T04:43:34.764777+0000","last_peered":"2023-08-25T04:43:34.764777+0000","last_clean":"2023-08-25T04:43:34.764777+0000","last_became_active":"2023-08-25T04:42:50.328645+0000","last_became_peered":"2023-08-25T04:42:50.328645+0000","last_unstale":"2023-08-25T04:43:34.764777+0000","last_undegraded":"2023-08-25T04:43:34.764777+0000","last_fullsized":"2023-08-25T04:43:34.764777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:25:29.024128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099012+0000","last_change":"2023-08-25T04:42:54.352024+0000","last_active":"2023-08-25T04:43:35.099012+0000","last_peered":"2023-08-25T04:43:35.099012+0000","last_clean":"2023-08-25T04:43:35.099012+0000","last_became_active":"2023-08-25T04:42:54.351306+0000","last_became_peered":"2023-08-25T04:42:54.351306+0000","last_unstale":"2023-08-25T04:43:35.099012+0000","last_undegraded":"2023-08-25T04:43:35.099012+0000","last_fullsized":"2023-08-25T04:43:35.099012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:17.119675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922819+0000","last_change":"2023-08-25T04:42:49.328425+0000","last_active":"2023-08-25T04:43:34.922819+0000","last_peered":"2023-08-25T04:43:34.922819+0000","last_clean":"2023-08-25T04:43:34.922819+0000","last_became_active":"2023-08-25T04:42:49.328112+0000","last_became_peered":"2023-08-25T04:42:49.328112+0000","last_unstale":"2023-08-25T04:43:34.922819+0000","last_undegraded":"2023-08-25T04:43:34.922819+0000","last_fullsized":"2023-08-25T04:43:34.922819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:42.375965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769924+0000","last_change":"2023-08-25T04:43:34.769924+0000","last_active":"2023-08-25T04:43:34.769924+0000","last_peered":"2023-08-25T04:43:34.769924+0000","last_clean":"2023-08-25T04:43:34.769924+0000","last_became_active":"2023-08-25T04:43:34.769575+0000","last_became_peered":"2023-08-25T04:43:34.769575+0000","last_unstale":"2023-08-25T04:43:34.769924+0000","last_undegraded":"2023-08-25T04:43:34.769924+0000","last_fullsized":"2023-08-25T04:43:34.769924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:32.013614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099091+0000","last_change":"2023-08-25T04:42:54.350733+0000","last_active":"2023-08-25T04:43:35.099091+0000","last_peered":"2023-08-25T04:43:35.099091+0000","last_clean":"2023-08-25T04:43:35.099091+0000","last_became_active":"2023-08-25T04:42:54.350344+0000","last_became_peered":"2023-08-25T04:42:54.350344+0000","last_unstale":"2023-08-25T04:43:35.099091+0000","last_undegraded":"2023-08-25T04:43:35.099091+0000","last_fullsized":"2023-08-25T04:43:35.099091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:32:02.438964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.772835+0000","last_change":"2023-08-25T04:43:34.771233+0000","last_active":"2023-08-25T04:43:34.772835+0000","last_peered":"2023-08-25T04:43:34.772835+0000","last_clean":"2023-08-25T04:43:34.772835+0000","last_became_active":"2023-08-25T04:43:34.770887+0000","last_became_peered":"2023-08-25T04:43:34.770887+0000","last_unstale":"2023-08-25T04:43:34.772835+0000","last_undegraded":"2023-08-25T04:43:34.772835+0000","last_fullsized":"2023-08-25T04:43:34.772835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:08.119078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":122,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:44:46.717700+0000","last_change":"2023-08-25T04:42:49.327294+0000","last_active":"2023-08-25T04:44:46.717700+0000","last_peered":"2023-08-25T04:44:46.717700+0000","last_clean":"2023-08-25T04:44:46.717700+0000","last_became_active":"2023-08-25T04:42:49.326982+0000","last_became_peered":"2023-08-25T04:42:49.326982+0000","last_unstale":"2023-08-25T04:44:46.717700+0000","last_undegraded":"2023-08-25T04:44:46.717700+0000","last_fullsized":"2023-08-25T04:44:46.717700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:43.271628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922975+0000","last_change":"2023-08-25T04:42:54.356196+0000","last_active":"2023-08-25T04:43:34.922975+0000","last_peered":"2023-08-25T04:43:34.922975+0000","last_clean":"2023-08-25T04:43:34.922975+0000","last_became_active":"2023-08-25T04:42:54.355708+0000","last_became_peered":"2023-08-25T04:42:54.355708+0000","last_unstale":"2023-08-25T04:43:34.922975+0000","last_undegraded":"2023-08-25T04:43:34.922975+0000","last_fullsized":"2023-08-25T04:43:34.922975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:04.252263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764688+0000","last_change":"2023-08-25T04:42:50.334130+0000","last_active":"2023-08-25T04:43:34.764688+0000","last_peered":"2023-08-25T04:43:34.764688+0000","last_clean":"2023-08-25T04:43:34.764688+0000","last_became_active":"2023-08-25T04:42:50.333843+0000","last_became_peered":"2023-08-25T04:42:50.333843+0000","last_unstale":"2023-08-25T04:43:34.764688+0000","last_undegraded":"2023-08-25T04:43:34.764688+0000","last_fullsized":"2023-08-25T04:43:34.764688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:33:20.016878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761743+0000","last_change":"2023-08-25T04:42:49.322826+0000","last_active":"2023-08-25T04:43:34.761743+0000","last_peered":"2023-08-25T04:43:34.761743+0000","last_clean":"2023-08-25T04:43:34.761743+0000","last_became_active":"2023-08-25T04:42:49.322509+0000","last_became_peered":"2023-08-25T04:42:49.322509+0000","last_unstale":"2023-08-25T04:43:34.761743+0000","last_undegraded":"2023-08-25T04:43:34.761743+0000","last_fullsized":"2023-08-25T04:43:34.761743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:10.259329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922646+0000","last_change":"2023-08-25T04:41:47.207105+0000","last_active":"2023-08-25T04:43:34.922646+0000","last_peered":"2023-08-25T04:43:34.922646+0000","last_clean":"2023-08-25T04:43:34.922646+0000","last_became_active":"2023-08-25T04:41:47.206678+0000","last_became_peered":"2023-08-25T04:41:47.206678+0000","last_unstale":"2023-08-25T04:43:34.922646+0000","last_undegraded":"2023-08-25T04:43:34.922646+0000","last_fullsized":"2023-08-25T04:43:34.922646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:38:48.217850+0000","last_clean_scrub_stamp":"2023-08-25T04:38:48.217850+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:39:20.569443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099295+0000","last_change":"2023-08-25T04:42:54.351089+0000","last_active":"2023-08-25T04:43:35.099295+0000","last_peered":"2023-08-25T04:43:35.099295+0000","last_clean":"2023-08-25T04:43:35.099295+0000","last_became_active":"2023-08-25T04:42:54.350484+0000","last_became_peered":"2023-08-25T04:42:54.350484+0000","last_unstale":"2023-08-25T04:43:35.099295+0000","last_undegraded":"2023-08-25T04:43:35.099295+0000","last_fullsized":"2023-08-25T04:43:35.099295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.332881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098261+0000","last_change":"2023-08-25T04:42:50.327335+0000","last_active":"2023-08-25T04:43:35.098261+0000","last_peered":"2023-08-25T04:43:35.098261+0000","last_clean":"2023-08-25T04:43:35.098261+0000","last_became_active":"2023-08-25T04:42:50.326992+0000","last_became_peered":"2023-08-25T04:42:50.326992+0000","last_unstale":"2023-08-25T04:43:35.098261+0000","last_undegraded":"2023-08-25T04:43:35.098261+0000","last_fullsized":"2023-08-25T04:43:35.098261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:51:31.902430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922785+0000","last_change":"2023-08-25T04:42:49.327021+0000","last_active":"2023-08-25T04:43:34.922785+0000","last_peered":"2023-08-25T04:43:34.922785+0000","last_clean":"2023-08-25T04:43:34.922785+0000","last_became_active":"2023-08-25T04:42:49.326717+0000","last_became_peered":"2023-08-25T04:42:49.326717+0000","last_unstale":"2023-08-25T04:43:34.922785+0000","last_undegraded":"2023-08-25T04:43:34.922785+0000","last_fullsized":"2023-08-25T04:43:34.922785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:47.704937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765952+0000","last_change":"2023-08-25T04:42:54.350287+0000","last_active":"2023-08-25T04:43:34.765952+0000","last_peered":"2023-08-25T04:43:34.765952+0000","last_clean":"2023-08-25T04:43:34.765952+0000","last_became_active":"2023-08-25T04:42:54.349883+0000","last_became_peered":"2023-08-25T04:42:54.349883+0000","last_unstale":"2023-08-25T04:43:34.765952+0000","last_undegraded":"2023-08-25T04:43:34.765952+0000","last_fullsized":"2023-08-25T04:43:34.765952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:38:36.745026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765218+0000","last_change":"2023-08-25T04:42:49.326531+0000","last_active":"2023-08-25T04:43:34.765218+0000","last_peered":"2023-08-25T04:43:34.765218+0000","last_clean":"2023-08-25T04:43:34.765218+0000","last_became_active":"2023-08-25T04:42:49.326211+0000","last_became_peered":"2023-08-25T04:42:49.326211+0000","last_unstale":"2023-08-25T04:43:34.765218+0000","last_undegraded":"2023-08-25T04:43:34.765218+0000","last_fullsized":"2023-08-25T04:43:34.765218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:15:24.598228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.326379+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.326076+0000","last_became_peered":"2023-08-25T04:42:50.326076+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:14:26.761777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764760+0000","last_change":"2023-08-25T04:42:54.357798+0000","last_active":"2023-08-25T04:43:34.764760+0000","last_peered":"2023-08-25T04:43:34.764760+0000","last_clean":"2023-08-25T04:43:34.764760+0000","last_became_active":"2023-08-25T04:42:54.357412+0000","last_became_peered":"2023-08-25T04:42:54.357412+0000","last_unstale":"2023-08-25T04:43:34.764760+0000","last_undegraded":"2023-08-25T04:43:34.764760+0000","last_fullsized":"2023-08-25T04:43:34.764760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:23.436529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.777294+0000","last_change":"2023-08-25T04:43:34.777294+0000","last_active":"2023-08-25T04:43:34.777294+0000","last_peered":"2023-08-25T04:43:34.777294+0000","last_clean":"2023-08-25T04:43:34.777294+0000","last_became_active":"2023-08-25T04:43:34.768036+0000","last_became_peered":"2023-08-25T04:43:34.768036+0000","last_unstale":"2023-08-25T04:43:34.777294+0000","last_undegraded":"2023-08-25T04:43:34.777294+0000","last_fullsized":"2023-08-25T04:43:34.777294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:22:52.521174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765705+0000","last_change":"2023-08-25T04:42:50.325303+0000","last_active":"2023-08-25T04:43:34.765705+0000","last_peered":"2023-08-25T04:43:34.765705+0000","last_clean":"2023-08-25T04:43:34.765705+0000","last_became_active":"2023-08-25T04:42:50.324975+0000","last_became_peered":"2023-08-25T04:42:50.324975+0000","last_unstale":"2023-08-25T04:43:34.765705+0000","last_undegraded":"2023-08-25T04:43:34.765705+0000","last_fullsized":"2023-08-25T04:43:34.765705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:34.577696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768990+0000","last_change":"2023-08-25T04:42:54.355600+0000","last_active":"2023-08-25T04:43:34.768990+0000","last_peered":"2023-08-25T04:43:34.768990+0000","last_clean":"2023-08-25T04:43:34.768990+0000","last_became_active":"2023-08-25T04:42:54.355212+0000","last_became_peered":"2023-08-25T04:42:54.355212+0000","last_unstale":"2023-08-25T04:43:34.768990+0000","last_undegraded":"2023-08-25T04:43:34.768990+0000","last_fullsized":"2023-08-25T04:43:34.768990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:09.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761921+0000","last_change":"2023-08-25T04:42:49.323657+0000","last_active":"2023-08-25T04:43:34.761921+0000","last_peered":"2023-08-25T04:43:34.761921+0000","last_clean":"2023-08-25T04:43:34.761921+0000","last_became_active":"2023-08-25T04:42:49.323360+0000","last_became_peered":"2023-08-25T04:42:49.323360+0000","last_unstale":"2023-08-25T04:43:34.761921+0000","last_undegraded":"2023-08-25T04:43:34.761921+0000","last_fullsized":"2023-08-25T04:43:34.761921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:16.228950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764889+0000","last_change":"2023-08-25T04:42:50.335067+0000","last_active":"2023-08-25T04:43:34.764889+0000","last_peered":"2023-08-25T04:43:34.764889+0000","last_clean":"2023-08-25T04:43:34.764889+0000","last_became_active":"2023-08-25T04:42:50.334788+0000","last_became_peered":"2023-08-25T04:42:50.334788+0000","last_unstale":"2023-08-25T04:43:34.764889+0000","last_undegraded":"2023-08-25T04:43:34.764889+0000","last_fullsized":"2023-08-25T04:43:34.764889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:14.043346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.768669+0000","last_change":"2023-08-25T04:43:34.768669+0000","last_active":"2023-08-25T04:43:34.768669+0000","last_peered":"2023-08-25T04:43:34.768669+0000","last_clean":"2023-08-25T04:43:34.768669+0000","last_became_active":"2023-08-25T04:43:34.768275+0000","last_became_peered":"2023-08-25T04:43:34.768275+0000","last_unstale":"2023-08-25T04:43:34.768669+0000","last_undegraded":"2023-08-25T04:43:34.768669+0000","last_fullsized":"2023-08-25T04:43:34.768669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:39:53.898292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,11],"acting":[4,8,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098506+0000","last_change":"2023-08-25T04:42:49.324352+0000","last_active":"2023-08-25T04:43:35.098506+0000","last_peered":"2023-08-25T04:43:35.098506+0000","last_clean":"2023-08-25T04:43:35.098506+0000","last_became_active":"2023-08-25T04:42:49.324059+0000","last_became_peered":"2023-08-25T04:42:49.324059+0000","last_unstale":"2023-08-25T04:43:35.098506+0000","last_undegraded":"2023-08-25T04:43:35.098506+0000","last_fullsized":"2023-08-25T04:43:35.098506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:19.654388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764296+0000","last_change":"2023-08-25T04:42:50.326167+0000","last_active":"2023-08-25T04:43:34.764296+0000","last_peered":"2023-08-25T04:43:34.764296+0000","last_clean":"2023-08-25T04:43:34.764296+0000","last_became_active":"2023-08-25T04:42:50.325848+0000","last_became_peered":"2023-08-25T04:42:50.325848+0000","last_unstale":"2023-08-25T04:43:34.764296+0000","last_undegraded":"2023-08-25T04:43:34.764296+0000","last_fullsized":"2023-08-25T04:43:34.764296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:11:04.726743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761549+0000","last_change":"2023-08-25T04:42:54.357525+0000","last_active":"2023-08-25T04:43:34.761549+0000","last_peered":"2023-08-25T04:43:34.761549+0000","last_clean":"2023-08-25T04:43:34.761549+0000","last_became_active":"2023-08-25T04:42:54.357160+0000","last_became_peered":"2023-08-25T04:42:54.357160+0000","last_unstale":"2023-08-25T04:43:34.761549+0000","last_undegraded":"2023-08-25T04:43:34.761549+0000","last_fullsized":"2023-08-25T04:43:34.761549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:48.355591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765742+0000","last_change":"2023-08-25T04:42:49.333225+0000","last_active":"2023-08-25T04:43:34.765742+0000","last_peered":"2023-08-25T04:43:34.765742+0000","last_clean":"2023-08-25T04:43:34.765742+0000","last_became_active":"2023-08-25T04:42:49.332426+0000","last_became_peered":"2023-08-25T04:42:49.332426+0000","last_unstale":"2023-08-25T04:43:34.765742+0000","last_undegraded":"2023-08-25T04:43:34.765742+0000","last_fullsized":"2023-08-25T04:43:34.765742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:12:04.352025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766384+0000","last_change":"2023-08-25T04:42:50.324470+0000","last_active":"2023-08-25T04:43:34.766384+0000","last_peered":"2023-08-25T04:43:34.766384+0000","last_clean":"2023-08-25T04:43:34.766384+0000","last_became_active":"2023-08-25T04:42:50.323951+0000","last_became_peered":"2023-08-25T04:42:50.323951+0000","last_unstale":"2023-08-25T04:43:34.766384+0000","last_undegraded":"2023-08-25T04:43:34.766384+0000","last_fullsized":"2023-08-25T04:43:34.766384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:02:40.882696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761618+0000","last_change":"2023-08-25T04:42:54.357815+0000","last_active":"2023-08-25T04:43:34.761618+0000","last_peered":"2023-08-25T04:43:34.761618+0000","last_clean":"2023-08-25T04:43:34.761618+0000","last_became_active":"2023-08-25T04:42:54.357373+0000","last_became_peered":"2023-08-25T04:42:54.357373+0000","last_unstale":"2023-08-25T04:43:34.761618+0000","last_undegraded":"2023-08-25T04:43:34.761618+0000","last_fullsized":"2023-08-25T04:43:34.761618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:31:23.594133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.779656+0000","last_change":"2023-08-25T04:43:34.779656+0000","last_active":"2023-08-25T04:43:34.779656+0000","last_peered":"2023-08-25T04:43:34.779656+0000","last_clean":"2023-08-25T04:43:34.779656+0000","last_became_active":"2023-08-25T04:43:34.770821+0000","last_became_peered":"2023-08-25T04:43:34.770821+0000","last_unstale":"2023-08-25T04:43:34.779656+0000","last_undegraded":"2023-08-25T04:43:34.779656+0000","last_fullsized":"2023-08-25T04:43:34.779656+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:08:31.402768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766129+0000","last_change":"2023-08-25T04:42:50.333907+0000","last_active":"2023-08-25T04:43:34.766129+0000","last_peered":"2023-08-25T04:43:34.766129+0000","last_clean":"2023-08-25T04:43:34.766129+0000","last_became_active":"2023-08-25T04:42:50.333518+0000","last_became_peered":"2023-08-25T04:42:50.333518+0000","last_unstale":"2023-08-25T04:43:34.766129+0000","last_undegraded":"2023-08-25T04:43:34.766129+0000","last_fullsized":"2023-08-25T04:43:34.766129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:18.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.761763+0000","last_change":"2023-08-25T04:42:54.359865+0000","last_active":"2023-08-25T04:43:34.761763+0000","last_peered":"2023-08-25T04:43:34.761763+0000","last_clean":"2023-08-25T04:43:34.761763+0000","last_became_active":"2023-08-25T04:42:54.359482+0000","last_became_peered":"2023-08-25T04:42:54.359482+0000","last_unstale":"2023-08-25T04:43:34.761763+0000","last_undegraded":"2023-08-25T04:43:34.761763+0000","last_fullsized":"2023-08-25T04:43:34.761763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:31:14.914419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765951+0000","last_change":"2023-08-25T04:42:49.324394+0000","last_active":"2023-08-25T04:43:34.765951+0000","last_peered":"2023-08-25T04:43:34.765951+0000","last_clean":"2023-08-25T04:43:34.765951+0000","last_became_active":"2023-08-25T04:42:49.324069+0000","last_became_peered":"2023-08-25T04:42:49.324069+0000","last_unstale":"2023-08-25T04:43:34.765951+0000","last_undegraded":"2023-08-25T04:43:34.765951+0000","last_fullsized":"2023-08-25T04:43:34.765951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:50:07.289831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.922875+0000","last_change":"2023-08-25T04:42:50.327873+0000","last_active":"2023-08-25T04:43:34.922875+0000","last_peered":"2023-08-25T04:43:34.922875+0000","last_clean":"2023-08-25T04:43:34.922875+0000","last_became_active":"2023-08-25T04:42:50.327569+0000","last_became_peered":"2023-08-25T04:42:50.327569+0000","last_unstale":"2023-08-25T04:43:34.922875+0000","last_undegraded":"2023-08-25T04:43:34.922875+0000","last_fullsized":"2023-08-25T04:43:34.922875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:24:13.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098664+0000","last_change":"2023-08-25T04:42:54.347773+0000","last_active":"2023-08-25T04:43:35.098664+0000","last_peered":"2023-08-25T04:43:35.098664+0000","last_clean":"2023-08-25T04:43:35.098664+0000","last_became_active":"2023-08-25T04:42:54.347384+0000","last_became_peered":"2023-08-25T04:42:54.347384+0000","last_unstale":"2023-08-25T04:43:35.098664+0000","last_undegraded":"2023-08-25T04:43:35.098664+0000","last_fullsized":"2023-08-25T04:43:35.098664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:02:41.742505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769588+0000","last_change":"2023-08-25T04:42:49.325260+0000","last_active":"2023-08-25T04:43:34.769588+0000","last_peered":"2023-08-25T04:43:34.769588+0000","last_clean":"2023-08-25T04:43:34.769588+0000","last_became_active":"2023-08-25T04:42:49.324748+0000","last_became_peered":"2023-08-25T04:42:49.324748+0000","last_unstale":"2023-08-25T04:43:34.769588+0000","last_undegraded":"2023-08-25T04:43:34.769588+0000","last_fullsized":"2023-08-25T04:43:34.769588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:56.254087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923325+0000","last_change":"2023-08-25T04:42:50.328684+0000","last_active":"2023-08-25T04:43:34.923325+0000","last_peered":"2023-08-25T04:43:34.923325+0000","last_clean":"2023-08-25T04:43:34.923325+0000","last_became_active":"2023-08-25T04:42:50.328239+0000","last_became_peered":"2023-08-25T04:42:50.328239+0000","last_unstale":"2023-08-25T04:43:34.923325+0000","last_undegraded":"2023-08-25T04:43:34.923325+0000","last_fullsized":"2023-08-25T04:43:34.923325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:27.560915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.764860+0000","last_change":"2023-08-25T04:42:54.358574+0000","last_active":"2023-08-25T04:43:34.764860+0000","last_peered":"2023-08-25T04:43:34.764860+0000","last_clean":"2023-08-25T04:43:34.764860+0000","last_became_active":"2023-08-25T04:42:54.358172+0000","last_became_peered":"2023-08-25T04:42:54.358172+0000","last_unstale":"2023-08-25T04:43:34.764860+0000","last_undegraded":"2023-08-25T04:43:34.764860+0000","last_fullsized":"2023-08-25T04:43:34.764860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:05.061144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099452+0000","last_change":"2023-08-25T04:42:49.327559+0000","last_active":"2023-08-25T04:43:35.099452+0000","last_peered":"2023-08-25T04:43:35.099452+0000","last_clean":"2023-08-25T04:43:35.099452+0000","last_became_active":"2023-08-25T04:42:49.327256+0000","last_became_peered":"2023-08-25T04:42:49.327256+0000","last_unstale":"2023-08-25T04:43:35.099452+0000","last_undegraded":"2023-08-25T04:43:35.099452+0000","last_fullsized":"2023-08-25T04:43:35.099452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:28.319336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923060+0000","last_change":"2023-08-25T04:42:50.332443+0000","last_active":"2023-08-25T04:43:34.923060+0000","last_peered":"2023-08-25T04:43:34.923060+0000","last_clean":"2023-08-25T04:43:34.923060+0000","last_became_active":"2023-08-25T04:42:50.332129+0000","last_became_peered":"2023-08-25T04:42:50.332129+0000","last_unstale":"2023-08-25T04:43:34.923060+0000","last_undegraded":"2023-08-25T04:43:34.923060+0000","last_fullsized":"2023-08-25T04:43:34.923060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:25.511917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098448+0000","last_change":"2023-08-25T04:42:54.355961+0000","last_active":"2023-08-25T04:43:35.098448+0000","last_peered":"2023-08-25T04:43:35.098448+0000","last_clean":"2023-08-25T04:43:35.098448+0000","last_became_active":"2023-08-25T04:42:54.355546+0000","last_became_peered":"2023-08-25T04:42:54.355546+0000","last_unstale":"2023-08-25T04:43:35.098448+0000","last_undegraded":"2023-08-25T04:43:35.098448+0000","last_fullsized":"2023-08-25T04:43:35.098448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:53.321989+0000","last_clean_scrub_stamp":"2023-08-25T04:42:53.321989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:15.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099187+0000","last_change":"2023-08-25T04:42:49.326908+0000","last_active":"2023-08-25T04:43:35.099187+0000","last_peered":"2023-08-25T04:43:35.099187+0000","last_clean":"2023-08-25T04:43:35.099187+0000","last_became_active":"2023-08-25T04:42:49.326492+0000","last_became_peered":"2023-08-25T04:42:49.326492+0000","last_unstale":"2023-08-25T04:43:35.099187+0000","last_undegraded":"2023-08-25T04:43:35.099187+0000","last_fullsized":"2023-08-25T04:43:35.099187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:48.300088+0000","last_clean_scrub_stamp":"2023-08-25T04:42:48.300088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:00.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.923329+0000","last_change":"2023-08-25T04:42:50.334596+0000","last_active":"2023-08-25T04:43:34.923329+0000","last_peered":"2023-08-25T04:43:34.923329+0000","last_clean":"2023-08-25T04:43:34.923329+0000","last_became_active":"2023-08-25T04:42:50.334195+0000","last_became_peered":"2023-08-25T04:42:50.334195+0000","last_unstale":"2023-08-25T04:43:34.923329+0000","last_undegraded":"2023-08-25T04:43:34.923329+0000","last_fullsized":"2023-08-25T04:43:34.923329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:54:03.933897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766391+0000","last_change":"2023-08-25T04:42:50.324934+0000","last_active":"2023-08-25T04:43:34.766391+0000","last_peered":"2023-08-25T04:43:34.766391+0000","last_clean":"2023-08-25T04:43:34.766391+0000","last_became_active":"2023-08-25T04:42:50.324607+0000","last_became_peered":"2023-08-25T04:42:50.324607+0000","last_unstale":"2023-08-25T04:43:34.766391+0000","last_undegraded":"2023-08-25T04:43:34.766391+0000","last_fullsized":"2023-08-25T04:43:34.766391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:42.835852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769164+0000","last_change":"2023-08-25T04:42:50.334824+0000","last_active":"2023-08-25T04:43:34.769164+0000","last_peered":"2023-08-25T04:43:34.769164+0000","last_clean":"2023-08-25T04:43:34.769164+0000","last_became_active":"2023-08-25T04:42:50.334554+0000","last_became_peered":"2023-08-25T04:42:50.334554+0000","last_unstale":"2023-08-25T04:43:34.769164+0000","last_undegraded":"2023-08-25T04:43:34.769164+0000","last_fullsized":"2023-08-25T04:43:34.769164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:38.408069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.769152+0000","last_change":"2023-08-25T04:42:50.330466+0000","last_active":"2023-08-25T04:43:34.769152+0000","last_peered":"2023-08-25T04:43:34.769152+0000","last_clean":"2023-08-25T04:43:34.769152+0000","last_became_active":"2023-08-25T04:42:50.330173+0000","last_became_peered":"2023-08-25T04:42:50.330173+0000","last_unstale":"2023-08-25T04:43:34.769152+0000","last_undegraded":"2023-08-25T04:43:34.769152+0000","last_fullsized":"2023-08-25T04:43:34.769152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:25.658427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.100100+0000","last_change":"2023-08-25T04:43:35.100100+0000","last_active":"2023-08-25T04:43:35.100100+0000","last_peered":"2023-08-25T04:43:35.100100+0000","last_clean":"2023-08-25T04:43:35.100100+0000","last_became_active":"2023-08-25T04:43:35.099734+0000","last_became_peered":"2023-08-25T04:43:35.099734+0000","last_unstale":"2023-08-25T04:43:35.100100+0000","last_undegraded":"2023-08-25T04:43:35.100100+0000","last_fullsized":"2023-08-25T04:43:35.100100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:18:17.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099291+0000","last_change":"2023-08-25T04:42:50.325967+0000","last_active":"2023-08-25T04:43:35.099291+0000","last_peered":"2023-08-25T04:43:35.099291+0000","last_clean":"2023-08-25T04:43:35.099291+0000","last_became_active":"2023-08-25T04:42:50.325634+0000","last_became_peered":"2023-08-25T04:42:50.325634+0000","last_unstale":"2023-08-25T04:43:35.099291+0000","last_undegraded":"2023-08-25T04:43:35.099291+0000","last_fullsized":"2023-08-25T04:43:35.099291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:05:39.229799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.766561+0000","last_change":"2023-08-25T04:42:50.331974+0000","last_active":"2023-08-25T04:43:34.766561+0000","last_peered":"2023-08-25T04:43:34.766561+0000","last_clean":"2023-08-25T04:43:34.766561+0000","last_became_active":"2023-08-25T04:42:50.331671+0000","last_became_peered":"2023-08-25T04:42:50.331671+0000","last_unstale":"2023-08-25T04:43:34.766561+0000","last_undegraded":"2023-08-25T04:43:34.766561+0000","last_fullsized":"2023-08-25T04:43:34.766561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:47:14.892189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.770202+0000","last_change":"2023-08-25T04:42:50.329628+0000","last_active":"2023-08-25T04:43:34.770202+0000","last_peered":"2023-08-25T04:43:34.770202+0000","last_clean":"2023-08-25T04:43:34.770202+0000","last_became_active":"2023-08-25T04:42:50.329316+0000","last_became_peered":"2023-08-25T04:42:50.329316+0000","last_unstale":"2023-08-25T04:43:34.770202+0000","last_undegraded":"2023-08-25T04:43:34.770202+0000","last_fullsized":"2023-08-25T04:43:34.770202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:32:31.428150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765901+0000","last_change":"2023-08-25T04:42:50.326253+0000","last_active":"2023-08-25T04:43:34.765901+0000","last_peered":"2023-08-25T04:43:34.765901+0000","last_clean":"2023-08-25T04:43:34.765901+0000","last_became_active":"2023-08-25T04:42:50.325963+0000","last_became_peered":"2023-08-25T04:42:50.325963+0000","last_unstale":"2023-08-25T04:43:34.765901+0000","last_undegraded":"2023-08-25T04:43:34.765901+0000","last_fullsized":"2023-08-25T04:43:34.765901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:05:04.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098709+0000","last_change":"2023-08-25T04:42:50.332007+0000","last_active":"2023-08-25T04:43:35.098709+0000","last_peered":"2023-08-25T04:43:35.098709+0000","last_clean":"2023-08-25T04:43:35.098709+0000","last_became_active":"2023-08-25T04:42:50.331644+0000","last_became_peered":"2023-08-25T04:42:50.331644+0000","last_unstale":"2023-08-25T04:43:35.098709+0000","last_undegraded":"2023-08-25T04:43:35.098709+0000","last_fullsized":"2023-08-25T04:43:35.098709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:13.517929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.765879+0000","last_change":"2023-08-25T04:42:50.329828+0000","last_active":"2023-08-25T04:43:34.765879+0000","last_peered":"2023-08-25T04:43:34.765879+0000","last_clean":"2023-08-25T04:43:34.765879+0000","last_became_active":"2023-08-25T04:42:50.329474+0000","last_became_peered":"2023-08-25T04:42:50.329474+0000","last_unstale":"2023-08-25T04:43:34.765879+0000","last_undegraded":"2023-08-25T04:43:34.765879+0000","last_fullsized":"2023-08-25T04:43:34.765879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:03:35.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.099213+0000","last_change":"2023-08-25T04:42:50.337207+0000","last_active":"2023-08-25T04:43:35.099213+0000","last_peered":"2023-08-25T04:43:35.099213+0000","last_clean":"2023-08-25T04:43:35.099213+0000","last_became_active":"2023-08-25T04:42:50.336905+0000","last_became_peered":"2023-08-25T04:42:50.336905+0000","last_unstale":"2023-08-25T04:43:35.099213+0000","last_undegraded":"2023-08-25T04:43:35.099213+0000","last_fullsized":"2023-08-25T04:43:35.099213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:15:06.086740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:35.098646+0000","last_change":"2023-08-25T04:42:50.324455+0000","last_active":"2023-08-25T04:43:35.098646+0000","last_peered":"2023-08-25T04:43:35.098646+0000","last_clean":"2023-08-25T04:43:35.098646+0000","last_became_active":"2023-08-25T04:42:50.324113+0000","last_became_peered":"2023-08-25T04:42:50.324113+0000","last_unstale":"2023-08-25T04:43:35.098646+0000","last_undegraded":"2023-08-25T04:43:35.098646+0000","last_fullsized":"2023-08-25T04:43:35.098646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:42.910594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-25T04:43:34.771119+0000","last_change":"2023-08-25T04:43:34.771119+0000","last_active":"2023-08-25T04:43:34.771119+0000","last_peered":"2023-08-25T04:43:34.771119+0000","last_clean":"2023-08-25T04:43:34.771119+0000","last_became_active":"2023-08-25T04:43:34.770717+0000","last_became_peered":"2023-08-25T04:43:34.770717+0000","last_unstale":"2023-08-25T04:43:34.771119+0000","last_undegraded":"2023-08-25T04:43:34.771119+0000","last_fullsized":"2023-08-25T04:43:34.771119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T04:42:49.303566+0000","last_clean_scrub_stamp":"2023-08-25T04:42:49.303566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:42.596435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":210752,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":576,"num_read_kb":302,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":483313,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":1.018,"15min":1.018},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90500000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70599999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71099999999999997,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.54700000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.496}]},{"osd":5,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84999999999999998,"5min":1.006,"15min":1.006},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.497,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.875,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.64800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":1.099,"15min":1.099},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.69999999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.66300000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63400000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79400000000000004}]},{"osd":9,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85099999999999998,"15min":0.84499999999999997},"min":{"1min":0.76300000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.85699999999999998,"15min":0.84599999999999997},"min":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.83199999999999996}]},{"osd":10,"last update":"Fri Aug 25 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.72099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97899999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72499999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1078519,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.45100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.40699999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89300000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91100000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.036,"15min":1.036},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.91000000000000003,"5min":1.167,"15min":1.167},"last":0.80000000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77700000000000002},"min":{"1min":0.64500000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.72399999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841580,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483549,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.61299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.432},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70399999999999996,"15min":0.72099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.745},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84899999999999998,"5min":1.266,"15min":1.266},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70599999999999996,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.73799999999999999,"15min":0.78200000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69799999999999995,"15min":0.73299999999999998},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.71899999999999997},"min":{"1min":0.41799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.628}]},{"osd":4,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80500000000000005,"15min":0.86499999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88200000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78800000000000003,"15min":0.82099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.85599999999999998},"min":{"1min":0.501,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:42:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.93799999999999994},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.87,"15min":0.92300000000000004},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.873,"15min":0.92700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.413,"15min":1.413},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.95999999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.47,"15min":1.47},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.876}]},{"osd":11,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81000000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103217,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483138,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.376,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60299999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.441,"15min":0.41999999999999998},"min":{"1min":0.34599999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.746,"15min":0.746},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8730000000000002,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.871}]},{"osd":4,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.754}]},{"osd":5,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83199999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.747,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93200000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:44:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70899999999999996,"15min":0.68400000000000005},"min":{"1min":0.57899999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Aug 25 04:44:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.82599999999999996,"15min":0.78100000000000003},"min":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.9119999999999999,"5min":4.9119999999999999,"15min":4.9119999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.83599999999999997,"15min":0.78800000000000003},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.9569999999999999,"5min":3.9569999999999999,"15min":3.9569999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.61399999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.75},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97399999999999998,"5min":1.024,"15min":1.024},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86199999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074818,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.5,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.41099999999999998},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.36899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.81599999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.58299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.876,"15min":0.876},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.628},"min":{"1min":0.51900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.71799999999999997,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.992,"5min":3.992,"15min":3.992},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95499999999999996,"5min":0.996,"15min":0.996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.623}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.8980000000000001,"5min":3.8980000000000001,"15min":3.8980000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":0.79800000000000004},"min":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.014,"15min":1.014},"last":0.67400000000000004}]},{"osd":10,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.78200000000000003,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.96599999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76200000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":482821,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.49299999999999999},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.57799999999999996,"15min":0.50900000000000001},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.347,"5min":3.347,"15min":3.347},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64800000000000002,"15min":0.59999999999999998},"min":{"1min":0.46899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68600000000000005,"15min":0.66100000000000003},"min":{"1min":0.53700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.745,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.69799999999999995,"15min":0.63100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":0.498},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.69299999999999995,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.625}]},{"osd":4,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.77700000000000002,"15min":0.72099999999999997},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.502,"5min":0.442,"15min":0.442},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77300000000000002,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0329999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.80700000000000005,"15min":0.78500000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.69499999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.84699999999999998,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60399999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.409,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.56299999999999994,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.1020000000000001,"5min":0.93000000000000005,"15min":0.90700000000000003},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.746}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.91300000000000003,"15min":0.90600000000000003},"min":{"1min":0.68700000000000006,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.82699999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.873},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.83999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":486300,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96299999999999997,"5min":5.2439999999999998,"15min":5.2439999999999998},"last":0.495}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.52600000000000002},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.94499999999999995,"5min":1.121,"15min":1.121},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59199999999999997,"15min":0.51200000000000001},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":1.548,"15min":1.548},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.42499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.401,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.59899999999999998},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.39800000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":1.044,"15min":1.044},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.436},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72299999999999998,"5min":1.153,"15min":1.153},"last":0.35599999999999998}]},{"osd":5,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96999999999999997,"5min":1.613,"15min":1.613},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69699999999999995,"15min":0.66400000000000003},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.2210000000000001,"5min":1.597,"15min":1.597},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.2130000000000001,"5min":1.26,"15min":1.26},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":0.75,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":11.375,"5min":11.375,"15min":11.375},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82899999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83999999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40799999999999997}]},{"osd":8,"last update":"Fri Aug 25 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Aug 25 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.76400000000000001,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.367,"5min":3.367,"15min":3.367},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.83499999999999996,"15min":0.79700000000000004},"min":{"1min":0.60599999999999998,"5min":0.437,"15min":0.437},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Aug 25 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.79400000000000004,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93000000000000005,"5min":1.302,"15min":1.302},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92100000000000004,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.79500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":484209,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 04:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.67300000000000004,"15min":0.54700000000000004},"min":{"1min":0.48699999999999999,"5min":0.216,"15min":0.216},"max":{"1min":4.5229999999999997,"5min":4.5229999999999997,"15min":4.5229999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.624,"15min":0.54600000000000004},"min":{"1min":0.51600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.53500000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.95699999999999996,"5min":1.194,"15min":1.194},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.63,"15min":0.57199999999999995},"min":{"1min":0.52100000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.499}]},{"osd":4,"last update":"Fri Aug 25 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.69299999999999995,"15min":0.60399999999999998},"min":{"1min":0.52000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.5960000000000001,"5min":4.5960000000000001,"15min":4.5960000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.48899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.68400000000000005,"15min":0.63400000000000001},"min":{"1min":0.56999999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94799999999999995}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.59799999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.028,"5min":1.365,"15min":1.365},"last":1.0029999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83399999999999996,"15min":0.82099999999999995},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.747},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.98799999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:44:45 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85799999999999998,"15min":0.82699999999999996},"min":{"1min":0.72099999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93899999999999995}]},{"osd":11,"last update":"Fri Aug 25 04:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.68799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95499999999999996,"15min":0.96299999999999997},"min":{"1min":0.73399999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.087,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.96499999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":488181,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55000000000000004,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.53000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Aug 25 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90500000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89500000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.622}]},{"osd":4,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.45100000000000001},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.45000000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Aug 25 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.61799999999999999},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96399999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63300000000000001,"15min":0.59199999999999997},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89900000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47899999999999998}]},{"osd":7,"last update":"Fri Aug 25 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Aug 25 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40600000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.622,"5min":0.44,"15min":0.44},"max":{"1min":0.98299999999999998,"5min":1.071,"15min":1.071},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.71299999999999997},"min":{"1min":0.59899999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99399999999999999,"5min":1.04,"15min":1.04},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.59999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80800000000000005}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.747},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83099999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483015,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.40100000000000002},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.42799999999999999},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.95099999999999996}]},{"osd":1,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.58299999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.341,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":1.53,"15min":1.53},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.573,"15min":1.573},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Aug 25 04:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.628},"min":{"1min":0.52000000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.4410000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Aug 25 04:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.73999999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.077,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.747,"15min":0.72499999999999998},"min":{"1min":0.57499999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.337,"15min":1.337},"last":0.621}]},{"osd":8,"last update":"Fri Aug 25 04:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Aug 25 04:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.85899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.495,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.67800000000000005}]},{"osd":10,"last update":"Fri Aug 25 04:43:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.88900000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.145},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1020000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66800000000000004}]},{"osd":11,"last update":"Fri Aug 25 04:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.64600000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":483375,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41399999999999998,"15min":0.36199999999999999},"min":{"1min":0.35999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":0.753,"15min":0.753},"last":0.374},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.376},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.501},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.625,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 25 04:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.51000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84999999999999998,"5min":1.99,"15min":1.99},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86299999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Aug 25 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.55700000000000005},"min":{"1min":0.53400000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.496,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88200000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98999999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri Aug 25 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.67600000000000005},"min":{"1min":0.69399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.65700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 25 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84799999999999998},"min":{"1min":0.64000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0880000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.099,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.73599999999999999}]},{"osd":10,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996}]},{"osd":11,"last update":"Fri Aug 25 04:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80200000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1074833,"data_compressed":53286,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.73299999999999998,"15min":0.59399999999999997},"min":{"1min":0.42099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":16.949999999999999,"5min":16.949999999999999,"15min":16.949999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.52600000000000002},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56499999999999995,"15min":0.50900000000000001},"min":{"1min":0.48299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.745}]},{"osd":2,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.375},"min":{"1min":0.33200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70799999999999996,"5min":0.755,"15min":0.755},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.40000000000000002},"min":{"1min":0.31900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67300000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 25 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.56200000000000006},"min":{"1min":0.51700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.876,"5min":0.88,"15min":0.88},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Aug 25 04:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.66600000000000004,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.8730000000000002,"5min":5.8730000000000002,"15min":5.8730000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.57299999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.60799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.88200000000000001}]},{"osd":8,"last update":"Fri Aug 25 04:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73999999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68100000000000005}]},{"osd":9,"last update":"Fri Aug 25 04:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.071},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.79900000000000004,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":5.8479999999999999,"5min":5.8479999999999999,"15min":5.8479999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Fri Aug 25 04:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.61599999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.72999999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35673,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T04:44:51.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:52.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.007+0000 7fd07a5ff700 1 -- 172.21.15.145:0/2617802381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 msgr2=0x7fd074108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.007+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/2617802381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd074108d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b0d0 tx=0x7fd06c00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 -- 172.21.15.145:0/2617802381 shutdown_connections 2023-08-25T04:44:52.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/2617802381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd074108d70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/2617802381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/2617802381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 0x7fd074101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 -- 172.21.15.145:0/2617802381 >> 172.21.15.145:0/2617802381 conn(0x7fd0740fbe60 msgr2=0x7fd0740fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:52.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 -- 172.21.15.145:0/2617802381 shutdown_connections 2023-08-25T04:44:52.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.008+0000 7fd07a5ff700 1 -- 172.21.15.145:0/2617802381 wait complete. 2023-08-25T04:44:52.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 Processor -- start 2023-08-25T04:44:52.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 -- start start 2023-08-25T04:44:52.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 0x7fd074198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd07419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741a0480 con 0x7fd0741024c0 2023-08-25T04:44:52.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741a05c0 con 0x7fd074101b20 2023-08-25T04:44:52.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd07a5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741a0750 con 0x7fd074100930 2023-08-25T04:44:52.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.009+0000 7fd073fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd07419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd073fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:36436/0 (socket says 172.21.15.145:36436) 2023-08-25T04:44:52.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd073fff700 1 -- 172.21.15.145:0/535137629 learned_addr learned my addr 172.21.15.145:0/535137629 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:52.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd0737fe700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 0x7fd074198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 -- 172.21.15.145:0/535137629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 msgr2=0x7fd074198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 -- 172.21.15.145:0/535137629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 msgr2=0x7fd074198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 0x7fd074198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 -- 172.21.15.145:0/535137629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c009d00 con 0x7fd0741024c0 2023-08-25T04:44:52.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd073fff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:52.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd078b9c700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd07419feb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd06c007a30 tx=0x7fd06c008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00e040 con 0x7fd0741024c0 2023-08-25T04:44:52.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06c00bd30 con 0x7fd0741024c0 2023-08-25T04:44:52.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0741a09d0 con 0x7fd0741024c0 2023-08-25T04:44:52.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.010+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c01d470 con 0x7fd0741024c0 2023-08-25T04:44:52.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.011+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0741a0ea0 con 0x7fd0741024c0 2023-08-25T04:44:52.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.012+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07410c1a0 con 0x7fd0741024c0 2023-08-25T04:44:52.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.013+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fd06c005020 con 0x7fd0741024c0 2023-08-25T04:44:52.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.015+0000 7fd0717fa700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 0x7fd05c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.015+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd06c0cc390 con 0x7fd0741024c0 2023-08-25T04:44:52.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.015+0000 7fd073fff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 0x7fd05c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.016+0000 7fd073fff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 0x7fd05c0a24e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd064005950 tx=0x7fd064009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.019+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd06c095eb0 con 0x7fd0741024c0 2023-08-25T04:44:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:51 smithi145 ceph-mon[97681]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:52.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:51 smithi162 ceph-mon[130941]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:51 smithi183 ceph-mon[131414]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:52.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.220+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd07410c390 con 0x7fd0741024c0 2023-08-25T04:44:52.225 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.222+0000 7fd0717fa700 1 -- 172.21.15.145:0/535137629 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7fd06c0994d0 con 0x7fd0741024c0 2023-08-25T04:44:52.226 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:52.226 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:52.227 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 msgr2=0x7fd05c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.227 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 0x7fd05c0a24e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd064005950 tx=0x7fd064009260 comp rx=0 tx=0).stop 2023-08-25T04:44:52.227 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 msgr2=0x7fd07419feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd07419feb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd06c007a30 tx=0x7fd06c008eb0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 shutdown_connections 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fd05c0a0020 0x7fd05c0a24e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0741024c0 0x7fd07419feb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd074100930 0x7fd074198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 --2- 172.21.15.145:0/535137629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd074101b20 0x7fd074198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 >> 172.21.15.145:0/535137629 conn(0x7fd0740fbe60 msgr2=0x7fd074107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:52.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 shutdown_connections 2023-08-25T04:44:52.229 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.225+0000 7fd07a5ff700 1 -- 172.21.15.145:0/535137629 wait complete. 2023-08-25T04:44:52.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T04:44:52.246 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 -- 172.21.15.145:0/1684592869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c101b20 msgr2=0x7fde0c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 --2- 172.21.15.145:0/1684592869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c101b20 0x7fde0c101f80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde000099c0 tx=0x7fde00009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 -- 172.21.15.145:0/1684592869 shutdown_connections 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 --2- 172.21.15.145:0/1684592869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c1024c0 0x7fde0c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 --2- 172.21.15.145:0/1684592869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c100930 0x7fde0c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 --2- 172.21.15.145:0/1684592869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c101b20 0x7fde0c101f80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 -- 172.21.15.145:0/1684592869 >> 172.21.15.145:0/1684592869 conn(0x7fde0c0fbea0 msgr2=0x7fde0c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:52.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 -- 172.21.15.145:0/1684592869 shutdown_connections 2023-08-25T04:44:52.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.445+0000 7fde10bff700 1 -- 172.21.15.145:0/1684592869 wait complete. 2023-08-25T04:44:52.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 Processor -- start 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 -- start start 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 0x7fde0c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c1024c0 0x7fde0c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0739f0 con 0x7fde0c100930 2023-08-25T04:44:52.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c073b60 con 0x7fde0c1024c0 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.446+0000 7fde10bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a0630 con 0x7fde0c101b20 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde09d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 0x7fde0c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54712/0 (socket says 172.21.15.145:54712) 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 -- 172.21.15.145:0/2655453432 learned_addr learned my addr 172.21.15.145:0/2655453432 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0ad9d700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c1024c0 0x7fde0c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 -- 172.21.15.145:0/2655453432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 msgr2=0x7fde0c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 0x7fde0c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 -- 172.21.15.145:0/2655453432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c1024c0 msgr2=0x7fde0c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c1024c0 0x7fde0c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 -- 172.21.15.145:0/2655453432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000095f0 con 0x7fde0c100930 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde09d9b700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 0x7fde0c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.447+0000 7fde0a59c700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e970 tx=0x7fddfc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.448+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc009790 con 0x7fde0c100930 2023-08-25T04:44:52.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.448+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddfc00c7e0 con 0x7fde0c100930 2023-08-25T04:44:52.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.448+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc003c70 con 0x7fde0c100930 2023-08-25T04:44:52.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.448+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c1a0910 con 0x7fde0c100930 2023-08-25T04:44:52.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.448+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c1a0dd0 con 0x7fde0c100930 2023-08-25T04:44:52.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.449+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c107f10 con 0x7fde0c100930 2023-08-25T04:44:52.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.451+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7fddfc00cc50 con 0x7fde0c100930 2023-08-25T04:44:52.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.452+0000 7fddfb7fe700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 0x7fddf40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.452+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fddfc014070 con 0x7fde0c100930 2023-08-25T04:44:52.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.452+0000 7fde09d9b700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 0x7fddf40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.453+0000 7fde09d9b700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 0x7fddf40a2410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde0000b580 tx=0x7fde0001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.456+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fddfc095140 con 0x7fde0c100930 2023-08-25T04:44:52.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.656+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde0c02cb70 con 0x7fde0c100930 2023-08-25T04:44:52.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.656+0000 7fddfb7fe700 1 -- 172.21.15.145:0/2655453432 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddfc003dd0 con 0x7fde0c100930 2023-08-25T04:44:52.658 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T04:44:52.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 msgr2=0x7fddf40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 0x7fddf40a2410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde0000b580 tx=0x7fde0001a040 comp rx=0 tx=0).stop 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 msgr2=0x7fde0c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e970 tx=0x7fddfc00ec80 comp rx=0 tx=0).stop 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 shutdown_connections 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fddf409ff50 0x7fddf40a2410 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0c1024c0 0x7fde0c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c100930 0x7fde0c0746d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 --2- 172.21.15.145:0/2655453432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde0c101b20 0x7fde0c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 >> 172.21.15.145:0/2655453432 conn(0x7fde0c0fbea0 msgr2=0x7fde0c105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:52.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 shutdown_connections 2023-08-25T04:44:52.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.659+0000 7fde10bff700 1 -- 172.21.15.145:0/2655453432 wait complete. 2023-08-25T04:44:52.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T04:44:52.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 -- 172.21.15.145:0/1849365388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 msgr2=0x7efe10108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 --2- 172.21.15.145:0/1849365388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe10108d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe08009f90 tx=0x7efe0800b330 comp rx=0 tx=0).stop 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 -- 172.21.15.145:0/1849365388 shutdown_connections 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 --2- 172.21.15.145:0/1849365388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe10108d70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 --2- 172.21.15.145:0/1849365388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10100930 0x7efe10100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 --2- 172.21.15.145:0/1849365388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10101b20 0x7efe10101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 -- 172.21.15.145:0/1849365388 >> 172.21.15.145:0/1849365388 conn(0x7efe100fbea0 msgr2=0x7efe100fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 -- 172.21.15.145:0/1849365388 shutdown_connections 2023-08-25T04:44:52.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.879+0000 7efe15dff700 1 -- 172.21.15.145:0/1849365388 wait complete. 2023-08-25T04:44:52.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.880+0000 7efe15dff700 1 Processor -- start 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 -- start start 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 0x7efe10196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe1019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019e250 con 0x7efe10101b20 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019e390 con 0x7efe10100930 2023-08-25T04:44:52.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe15dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019e520 con 0x7efe101024c0 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe0f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.881+0000 7efe0effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 0x7efe10196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60952/0 (socket says 172.21.15.145:60952) 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0f7fe700 1 -- 172.21.15.145:0/3412924186 learned_addr learned my addr 172.21.15.145:0/3412924186 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0ffff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe1019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 -- 172.21.15.145:0/3412924186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 msgr2=0x7efe1019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.886 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe1019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 -- 172.21.15.145:0/3412924186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 msgr2=0x7efe101966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 -- 172.21.15.145:0/3412924186 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08009c40 con 0x7efe10101b20 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0f7fe700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0effd700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 0x7efe10196c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e9f0 tx=0x7efe0400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400c9c0 con 0x7efe10101b20 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1019e800 con 0x7efe10101b20 2023-08-25T04:44:52.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe04010430 con 0x7efe10101b20 2023-08-25T04:44:52.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.882+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04018680 con 0x7efe10101b20 2023-08-25T04:44:52.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.883+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1019ed60 con 0x7efe10101b20 2023-08-25T04:44:52.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.883+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10107e70 con 0x7efe10101b20 2023-08-25T04:44:52.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.886+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7efe04010860 con 0x7efe10101b20 2023-08-25T04:44:52.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.887+0000 7efe0cff9700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 0x7efdf80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:52.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.887+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efe04014070 con 0x7efe10101b20 2023-08-25T04:44:52.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.887+0000 7efe0f7fe700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 0x7efdf80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:52.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.888+0000 7efe0f7fe700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 0x7efdf80a2490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe00009fd0 tx=0x7efe00009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:52.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:52.889+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7efe040951d0 con 0x7efe10101b20 2023-08-25T04:44:53.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:52 smithi145 ceph-mon[97681]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:52 smithi145 ceph-mon[97681]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:44:53.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:52 smithi145 ceph-mon[97681]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/535137629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:53.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:52 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2655453432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:53.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.091+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe1010c380 con 0x7efe10101b20 2023-08-25T04:44:53.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.092+0000 7efe0cff9700 1 -- 172.21.15.145:0/3412924186 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe040987f0 con 0x7efe10101b20 2023-08-25T04:44:53.094 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:53.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 msgr2=0x7efdf80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 0x7efdf80a2490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe00009fd0 tx=0x7efe00009310 comp rx=0 tx=0).stop 2023-08-25T04:44:53.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 msgr2=0x7efe10196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 0x7efe10196c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e9f0 tx=0x7efe0400ed00 comp rx=0 tx=0).stop 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 shutdown_connections 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7efdf809ffd0 0x7efdf80a2490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101024c0 0x7efe1019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe10100930 0x7efe101966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 --2- 172.21.15.145:0/3412924186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe10101b20 0x7efe10196c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 >> 172.21.15.145:0/3412924186 conn(0x7efe100fbea0 msgr2=0x7efe10104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:53.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.094+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 shutdown_connections 2023-08-25T04:44:53.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.095+0000 7efe15dff700 1 -- 172.21.15.145:0/3412924186 wait complete. 2023-08-25T04:44:53.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T04:44:53.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:52 smithi162 ceph-mon[130941]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:52 smithi162 ceph-mon[130941]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:44:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:52 smithi162 ceph-mon[130941]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/535137629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:53.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:52 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2655453432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:53.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:52 smithi183 ceph-mon[131414]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:52 smithi183 ceph-mon[131414]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:44:53.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:52 smithi183 ceph-mon[131414]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-25T04:44:53.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/535137629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:53.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:52 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2655453432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T04:44:53.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3346087848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec1024c0 msgr2=0x7f3bec108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3346087848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec1024c0 0x7f3bec108d70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b0d0 tx=0x7f3be400b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3346087848 shutdown_connections 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3346087848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec1024c0 0x7f3bec108d70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3346087848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3346087848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec101b20 0x7f3bec101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3346087848 >> 172.21.15.145:0/3346087848 conn(0x7f3bec0fbea0 msgr2=0x7f3bec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3346087848 shutdown_connections 2023-08-25T04:44:53.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.316+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3346087848 wait complete. 2023-08-25T04:44:53.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.317+0000 7f3bf17ff700 1 Processor -- start 2023-08-25T04:44:53.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.317+0000 7f3bf17ff700 1 -- start start 2023-08-25T04:44:53.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec1024c0 0x7f3bec19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a04d0 con 0x7f3bec101b20 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a0610 con 0x7f3bec100930 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bf17ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1a07a0 con 0x7f3bec1024c0 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57102/0 (socket says 172.21.15.145:57102) 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 -- 172.21.15.145:0/3035814963 learned_addr learned my addr 172.21.15.145:0/3035814963 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3beaffd700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3beb7fe700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec1024c0 0x7f3bec19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 -- 172.21.15.145:0/3035814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec1024c0 msgr2=0x7f3bec19ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec1024c0 0x7f3bec19ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 -- 172.21.15.145:0/3035814963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 msgr2=0x7f3bec198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3bea7fc700 1 -- 172.21.15.145:0/3035814963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4009d00 con 0x7f3bec101b20 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.318+0000 7f3beaffd700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:53.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bea7fc700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9a0 tx=0x7f3be000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:53.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009790 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3be000cbd0 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0018680 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec1a0a80 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.319+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec1a0f90 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.320+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec192f90 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.322+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3be000cd40 con 0x7f3bec101b20 2023-08-25T04:44:53.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.324+0000 7f3bd3fff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 0x7f3bd40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.324+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3be0014070 con 0x7f3bec101b20 2023-08-25T04:44:53.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.324+0000 7f3beaffd700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 0x7f3bd40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.324+0000 7f3beaffd700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 0x7f3bd40a2490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3bec101980 tx=0x7f3bdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:53.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.327+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3be0051e00 con 0x7f3bec101b20 2023-08-25T04:44:53.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.531+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bec02cdc0 con 0x7f3bec101b20 2023-08-25T04:44:53.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.531+0000 7f3bd3fff700 1 -- 172.21.15.145:0/3035814963 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3be00951e0 con 0x7f3bec101b20 2023-08-25T04:44:53.534 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T04:44:53.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 msgr2=0x7f3bd40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 0x7f3bd40a2490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3bec101980 tx=0x7f3bdc009260 comp rx=0 tx=0).stop 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 msgr2=0x7f3bec198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9a0 tx=0x7f3be000ecb0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 shutdown_connections 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3bd409ffd0 0x7f3bd40a2490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bec1024c0 0x7f3bec19ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bec100930 0x7f3bec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 --2- 172.21.15.145:0/3035814963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bec101b20 0x7f3bec198e90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 >> 172.21.15.145:0/3035814963 conn(0x7f3bec0fbea0 msgr2=0x7f3bec1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:53.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 shutdown_connections 2023-08-25T04:44:53.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.534+0000 7f3bf17ff700 1 -- 172.21.15.145:0/3035814963 wait complete. 2023-08-25T04:44:53.555 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T04:44:53.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 -- 172.21.15.145:0/1921048819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2594101b20 msgr2=0x7f2594101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 --2- 172.21.15.145:0/1921048819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2594101b20 0x7f2594101f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2584009990 tx=0x7f2584009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 -- 172.21.15.145:0/1921048819 shutdown_connections 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 --2- 172.21.15.145:0/1921048819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25941024c0 0x7f2594108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 --2- 172.21.15.145:0/1921048819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 --2- 172.21.15.145:0/1921048819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2594101b20 0x7f2594101f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 -- 172.21.15.145:0/1921048819 >> 172.21.15.145:0/1921048819 conn(0x7f25940fbea0 msgr2=0x7f25940fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.755+0000 7f259b3ff700 1 -- 172.21.15.145:0/1921048819 shutdown_connections 2023-08-25T04:44:53.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.756+0000 7f259b3ff700 1 -- 172.21.15.145:0/1921048819 wait complete. 2023-08-25T04:44:53.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.756+0000 7f259b3ff700 1 Processor -- start 2023-08-25T04:44:53.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 -- start start 2023-08-25T04:44:53.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594101b20 0x7f2594198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941a03a0 con 0x7f2594100930 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941a04e0 con 0x7f2594101b20 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259b3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941a0670 con 0x7f25941024c0 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57118/0 (socket says 172.21.15.145:57118) 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:39568/0 (socket says 172.21.15.145:39568) 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259919b700 1 -- 172.21.15.145:0/3363103503 learned_addr learned my addr 172.21.15.145:0/3363103503 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.757+0000 7f259899a700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594101b20 0x7f2594198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 -- 172.21.15.145:0/3363103503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594101b20 msgr2=0x7f2594198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594101b20 0x7f2594198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 -- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 msgr2=0x7f2594198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 -- 172.21.15.145:0/3363103503 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009520 con 0x7f25941024c0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259919b700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259999c700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258c00efc0 tx=0x7f258c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c018070 con 0x7f25941024c0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258c00f040 con 0x7f25941024c0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840095f0 con 0x7f25941024c0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00cd40 con 0x7f25941024c0 2023-08-25T04:44:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.758+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25941a0c20 con 0x7f25941024c0 2023-08-25T04:44:53.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.759+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2594192f80 con 0x7f25941024c0 2023-08-25T04:44:53.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.761+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f258c003a00 con 0x7f25941024c0 2023-08-25T04:44:53.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.763+0000 7f258a7fc700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 0x7f25800a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:53.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.763+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f258c0cbac0 con 0x7f25941024c0 2023-08-25T04:44:53.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.763+0000 7f259919b700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 0x7f25800a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:53.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.763+0000 7f259919b700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 0x7f25800a2410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2590009fd0 tx=0x7f2590009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:53.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.765+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f258c095580 con 0x7f25941024c0 2023-08-25T04:44:53.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.967+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f259410c320 con 0x7f25941024c0 2023-08-25T04:44:53.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.968+0000 7f258a7fc700 1 -- 172.21.15.145:0/3363103503 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f258c098ba0 con 0x7f25941024c0 2023-08-25T04:44:53.970 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T04:44:53.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 msgr2=0x7f25800a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 0x7f25800a2410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2590009fd0 tx=0x7f2590009260 comp rx=0 tx=0).stop 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 msgr2=0x7f259419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258c00efc0 tx=0x7f258c00c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 shutdown_connections 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f258009ff50 0x7f25800a2410 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25941024c0 0x7f259419fdd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2594100930 0x7f2594198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 --2- 172.21.15.145:0/3363103503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594101b20 0x7f2594198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:53.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 >> 172.21.15.145:0/3363103503 conn(0x7f25940fbea0 msgr2=0x7f2594107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:53.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 shutdown_connections 2023-08-25T04:44:53.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:53.971+0000 7f259b3ff700 1 -- 172.21.15.145:0/3363103503 wait complete. 2023-08-25T04:44:53.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:44:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:53 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3412924186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:53 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3035814963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:53 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3412924186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:53 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3035814963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:53 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3412924186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:53 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3035814963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T04:44:54.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 -- 172.21.15.145:0/802400124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c41087e0 msgr2=0x7f84c410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/802400124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c41087e0 0x7f84c410ac70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b0a0 tx=0x7f84bc00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 -- 172.21.15.145:0/802400124 shutdown_connections 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/802400124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c41087e0 0x7f84c410ac70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/802400124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c4074960 0x7f84c4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/802400124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4073380 0x7f84c4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 -- 172.21.15.145:0/802400124 >> 172.21.15.145:0/802400124 conn(0x7f84c40fbe50 msgr2=0x7f84c40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 -- 172.21.15.145:0/802400124 shutdown_connections 2023-08-25T04:44:54.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.187+0000 7f84cc7ff700 1 -- 172.21.15.145:0/802400124 wait complete. 2023-08-25T04:44:54.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.188+0000 7f84cc7ff700 1 Processor -- start 2023-08-25T04:44:54.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 -- start start 2023-08-25T04:44:54.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4074960 0x7f84c4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c41087e0 0x7f84c419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41a0460 con 0x7f84c4073380 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41a05a0 con 0x7f84c41087e0 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cc7ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41a0730 con 0x7f84c4074960 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57136/0 (socket says 172.21.15.145:57136) 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 -- 172.21.15.145:0/1387016962 learned_addr learned my addr 172.21.15.145:0/1387016962 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:54.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84c9d9a700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4074960 0x7f84c4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84cad9c700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c41087e0 0x7f84c419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 -- 172.21.15.145:0/1387016962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4074960 msgr2=0x7f84c4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4074960 0x7f84c4198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 -- 172.21.15.145:0/1387016962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c41087e0 msgr2=0x7f84c419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c41087e0 0x7f84c419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.189+0000 7f84ca59b700 1 -- 172.21.15.145:0/1387016962 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc009d00 con 0x7f84c4073380 2023-08-25T04:44:54.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84ca59b700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e9a0 tx=0x7f84c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0009790 con 0x7f84c4073380 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c0010430 con 0x7f84c4073380 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c41a0a10 con 0x7f84c4073380 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0018680 con 0x7f84c4073380 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.190+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41a0f70 con 0x7f84c4073380 2023-08-25T04:44:54.195 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.191+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c4106e90 con 0x7f84c4073380 2023-08-25T04:44:54.196 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.193+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f84c00105a0 con 0x7f84c4073380 2023-08-25T04:44:54.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.195+0000 7f84bb7fe700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 0x7f84b00a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.196+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f84c0014070 con 0x7f84c4073380 2023-08-25T04:44:54.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.196+0000 7f84c9d9a700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 0x7f84b00a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.196+0000 7f84c9d9a700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 0x7f84b00a24e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f84c4074680 tx=0x7f84b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:54.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.199+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f84c0051e00 con 0x7f84c4073380 2023-08-25T04:44:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.447+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c410e360 con 0x7f84c4073380 2023-08-25T04:44:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.448+0000 7f84bb7fe700 1 -- 172.21.15.145:0/1387016962 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f84c0095100 con 0x7f84c4073380 2023-08-25T04:44:54.449 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:54.450 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:44:54.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 msgr2=0x7f84b00a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 0x7f84b00a24e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f84c4074680 tx=0x7f84b4009260 comp rx=0 tx=0).stop 2023-08-25T04:44:54.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 msgr2=0x7f84c41988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e9a0 tx=0x7f84c000ecb0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 shutdown_connections 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f84b00a0020 0x7f84b00a24e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84c41087e0 0x7f84c419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84c4073380 0x7f84c41988e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 --2- 172.21.15.145:0/1387016962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c4074960 0x7f84c4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.450+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 >> 172.21.15.145:0/1387016962 conn(0x7f84c40fbe50 msgr2=0x7f84c40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.451+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 shutdown_connections 2023-08-25T04:44:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.451+0000 7f84cc7ff700 1 -- 172.21.15.145:0/1387016962 wait complete. 2023-08-25T04:44:54.454 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:44:54.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T04:44:54.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 -- 172.21.15.145:0/3472394959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b54101b20 msgr2=0x7f4b54101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 --2- 172.21.15.145:0/3472394959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b54101b20 0x7f4b54101f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4b400099c0 tx=0x7f4b40009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 -- 172.21.15.145:0/3472394959 shutdown_connections 2023-08-25T04:44:54.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 --2- 172.21.15.145:0/3472394959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b541024c0 0x7f4b54108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 --2- 172.21.15.145:0/3472394959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54100930 0x7f4b54100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 --2- 172.21.15.145:0/3472394959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b54101b20 0x7f4b54101f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 -- 172.21.15.145:0/3472394959 >> 172.21.15.145:0/3472394959 conn(0x7f4b540fbe60 msgr2=0x7f4b540fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:54.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 -- 172.21.15.145:0/3472394959 shutdown_connections 2023-08-25T04:44:54.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.669+0000 7f4b5adff700 1 -- 172.21.15.145:0/3472394959 wait complete. 2023-08-25T04:44:54.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 Processor -- start 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 -- start start 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54101b20 0x7f4b54198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b541024c0 0x7f4b5419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541a0380 con 0x7f4b54101b20 2023-08-25T04:44:54.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541a04c0 con 0x7f4b541024c0 2023-08-25T04:44:54.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b5adff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541a0650 con 0x7f4b54100930 2023-08-25T04:44:54.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.670+0000 7f4b58b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:39624/0 (socket says 172.21.15.145:39624) 2023-08-25T04:44:54.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 -- 172.21.15.145:0/2636145651 learned_addr learned my addr 172.21.15.145:0/2636145651 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b53fff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54101b20 0x7f4b54198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b5939c700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b541024c0 0x7f4b5419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 -- 172.21.15.145:0/2636145651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b541024c0 msgr2=0x7f4b5419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b541024c0 0x7f4b5419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 -- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54101b20 msgr2=0x7f4b54198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54101b20 0x7f4b54198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 -- 172.21.15.145:0/2636145651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009520 con 0x7f4b54100930 2023-08-25T04:44:54.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b58b9b700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800ea50 tx=0x7f4b4800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:54.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800cbd0 con 0x7f4b54100930 2023-08-25T04:44:54.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4800cd30 con 0x7f4b54100930 2023-08-25T04:44:54.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48018680 con 0x7f4b54100930 2023-08-25T04:44:54.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.671+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b400095f0 con 0x7f4b54100930 2023-08-25T04:44:54.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.672+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b541a0c60 con 0x7f4b54100930 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.672+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5410c190 con 0x7f4b54100930 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.675+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f4b48010430 con 0x7f4b54100930 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.676+0000 7f4b51ffb700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 0x7f4b440a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.676+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4b48014070 con 0x7f4b54100930 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.676+0000 7f4b53fff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 0x7f4b440a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:54.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.677+0000 7f4b53fff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 0x7f4b440a2460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b580 tx=0x7f4b40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:54.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.678+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4b48095670 con 0x7f4b54100930 2023-08-25T04:44:54.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.881+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b5410c3d0 con 0x7f4b54100930 2023-08-25T04:44:54.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.884+0000 7f4b51ffb700 1 -- 172.21.15.145:0/2636145651 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24447 (secure 0 0 0) 0x7f4b48098c90 con 0x7f4b54100930 2023-08-25T04:44:54.886 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:44:54.886 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":89,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T04:43:34.757361+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T04:44:54.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 msgr2=0x7f4b440a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 0x7f4b440a2460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b580 tx=0x7f4b40005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 msgr2=0x7f4b54198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:54.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800ea50 tx=0x7f4b4800ed60 comp rx=0 tx=0).stop 2023-08-25T04:44:54.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 shutdown_connections 2023-08-25T04:44:54.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f4b4409ffa0 0x7f4b440a2460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b541024c0 0x7f4b5419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100930 0x7f4b54198800 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 --2- 172.21.15.145:0/2636145651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b54101b20 0x7f4b54198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 >> 172.21.15.145:0/2636145651 conn(0x7f4b540fbe60 msgr2=0x7f4b540fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 shutdown_connections 2023-08-25T04:44:54.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:54.886+0000 7f4b5adff700 1 -- 172.21.15.145:0/2636145651 wait complete. 2023-08-25T04:44:54.907 DEBUG:tasks.fwd_scrub:Yielding 2023-08-25T04:44:54.908 INFO:teuthology.run_tasks:Running task exec... 2023-08-25T04:44:54.921 INFO:teuthology.task.exec:Executing custom commands... 2023-08-25T04:44:54.922 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:44:54.922 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-08-25T04:44:54.923 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-08-25T04:44:54.923 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-08-25T04:44:54.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-08-25T04:44:55.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:54 smithi145 ceph-mon[97681]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:44:55.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3363103503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:55.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:54 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1387016962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:55.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 -- 172.21.15.145:0/2574889831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1088100950 msgr2=0x7f1088100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 --2- 172.21.15.145:0/2574889831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1088100950 0x7f1088100d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f107c0099c0 tx=0x7f107c009cd0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 -- 172.21.15.145:0/2574889831 shutdown_connections 2023-08-25T04:44:55.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 --2- 172.21.15.145:0/2574889831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10881024e0 0x7f1088108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 --2- 172.21.15.145:0/2574889831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1088100950 0x7f1088100d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 --2- 172.21.15.145:0/2574889831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088101b40 0x7f1088101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.123+0000 7f108efff700 1 -- 172.21.15.145:0/2574889831 >> 172.21.15.145:0/2574889831 conn(0x7f10880fbea0 msgr2=0x7f10880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.124+0000 7f108efff700 1 -- 172.21.15.145:0/2574889831 shutdown_connections 2023-08-25T04:44:55.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.124+0000 7f108efff700 1 -- 172.21.15.145:0/2574889831 wait complete. 2023-08-25T04:44:55.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.124+0000 7f108efff700 1 Processor -- start 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 -- start start 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088100950 0x7f1088198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1088101b40 0x7f1088198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108819a340 con 0x7f10881024e0 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108819a4b0 con 0x7f1088101b40 2023-08-25T04:44:55.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108efff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108819a670 con 0x7f1088100950 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108d59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108d59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57174/0 (socket says 172.21.15.145:57174) 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108d59c700 1 -- 172.21.15.145:0/75999066 learned_addr learned my addr 172.21.15.145:0/75999066 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f1087fff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1088101b40 0x7f1088198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.125+0000 7f108cd9b700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088100950 0x7f1088198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 -- 172.21.15.145:0/75999066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088100950 msgr2=0x7f1088198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088100950 0x7f1088198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 -- 172.21.15.145:0/75999066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1088101b40 msgr2=0x7f1088198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1088101b40 0x7f1088198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 -- 172.21.15.145:0/75999066 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0095f0 con 0x7f10881024e0 2023-08-25T04:44:55.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108d59c700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f108000ef30 tx=0x7f108000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080018070 con 0x7f10881024e0 2023-08-25T04:44:55.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108819a8f0 con 0x7f10881024e0 2023-08-25T04:44:55.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108000f040 con 0x7f10881024e0 2023-08-25T04:44:55.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000cd00 con 0x7f10881024e0 2023-08-25T04:44:55.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.126+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10881a1110 con 0x7f10881024e0 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.127+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108804e700 con 0x7f10881024e0 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.130+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f1080003a00 con 0x7f10881024e0 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.131+0000 7f1085ffb700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 0x7f10780a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.131+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f10800cb0d0 con 0x7f10881024e0 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.131+0000 7f108cd9b700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 0x7f10780a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.132+0000 7f108cd9b700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 0x7f10780a2410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f107c009990 tx=0x7f107c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/3788977057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 msgr2=0x7f3b94101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/3788977057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94101f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009920 tx=0x7f3b80009c30 comp rx=0 tx=0).stop 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/3788977057 shutdown_connections 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/3788977057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b941024a0 0x7f3b94108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/3788977057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b94100910 0x7f3b94100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/3788977057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94101f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/3788977057 >> 172.21.15.145:0/3788977057 conn(0x7f3b940fbe60 msgr2=0x7f3b940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/3788977057 shutdown_connections 2023-08-25T04:44:55.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.134+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/3788977057 wait complete. 2023-08-25T04:44:55.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.135+0000 7f3b9b1ff700 1 Processor -- start 2023-08-25T04:44:55.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.135+0000 7f3b9b1ff700 1 -- start start 2023-08-25T04:44:55.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100910 0x7f3b94196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b941024a0 0x7f3b9419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419e1e0 con 0x7f3b941024a0 2023-08-25T04:44:55.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419e320 con 0x7f3b94101b00 2023-08-25T04:44:55.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9b1ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419e4b0 con 0x7f3b94100910 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b9979c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b941024a0 0x7f3b9419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100910 0x7f3b94196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46012/0 (socket says 172.21.15.145:46012) 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f3b93fff700 1 -- 172.21.15.145:0/1455305687 learned_addr learned my addr 172.21.15.145:0/1455305687 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.136+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f1080094b90 con 0x7f10881024e0 2023-08-25T04:44:55.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 -- 172.21.15.145:0/1455305687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100910 msgr2=0x7f3b94196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100910 0x7f3b94196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 -- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b941024a0 msgr2=0x7f3b9419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b941024a0 0x7f3b9419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 -- 172.21.15.145:0/1455305687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80009520 con 0x7f3b94101b00 2023-08-25T04:44:55.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b93fff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009230 tx=0x7f3b8000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b91ffb700 1 -- 172.21.15.145:0/1455305687 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8001c070 con 0x7f3b94101b00 2023-08-25T04:44:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9419e730 con 0x7f3b94101b00 2023-08-25T04:44:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b91ffb700 1 -- 172.21.15.145:0/1455305687 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b80005040 con 0x7f3b94101b00 2023-08-25T04:44:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b91ffb700 1 -- 172.21.15.145:0/1455305687 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80003a20 con 0x7f3b94101b00 2023-08-25T04:44:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.137+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9419eb50 con 0x7f3b94101b00 2023-08-25T04:44:55.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.142+0000 7f3b91ffb700 1 -- 172.21.15.145:0/1455305687 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f3b80020030 con 0x7f3b94101b00 2023-08-25T04:44:55.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 Processor -- start 2023-08-25T04:44:55.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 -- start start 2023-08-25T04:44:55.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3423e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3424dd0 0x558fd3425250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd338b830 con 0x558fd3424dd0 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd337c800 con 0x558fd34243b0 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.143+0000 7f3baec02b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd337a120 con 0x558fd3423a50 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b91ffb700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 0x7f3b840a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46028/0 (socket says 172.21.15.145:46028) 2023-08-25T04:44:55.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 -- 172.21.15.145:0/2963953461 learned_addr learned my addr 172.21.15.145:0/2963953461 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b91ffb700 1 -- 172.21.15.145:0/1455305687 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3b800cc660 con 0x7f3b94101b00 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b93fff700 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3424dd0 0x558fd3425250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b9979c700 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3423e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b9979c700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 0x7f3b840a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 -- 172.21.15.145:0/2963953461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 msgr2=0x558fd3423e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3423e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 -- 172.21.15.145:0/2963953461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3424dd0 msgr2=0x558fd3425250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3424dd0 0x558fd3425250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.144+0000 7f3b98f9b700 1 -- 172.21.15.145:0/2963953461 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd31f56b0 con 0x558fd34243b0 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.145+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009d30 tx=0x7f3b88011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b881a8dfaf0b1b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.145+0000 7f3b9979c700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 0x7f3b840a2410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009ec0 tx=0x7f3b8c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.145+0000 7f3b7effd700 1 -- 172.21.15.145:0/2963953461 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004220 con 0x558fd34243b0 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.145+0000 7f3b7effd700 1 -- 172.21.15.145:0/2963953461 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b88004cf0 con 0x558fd34243b0 2023-08-25T04:44:55.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.145+0000 7f3b7effd700 1 -- 172.21.15.145:0/2963953461 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88005540 con 0x558fd34243b0 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 -- 172.21.15.145:0/2963953461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 msgr2=0x558fd34247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009d30 tx=0x7f3b88011720 comp rx=0 tx=0).stop 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 -- 172.21.15.145:0/2963953461 shutdown_connections 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3424dd0 0x558fd3425250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3423e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2963953461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd34243b0 0x558fd34247f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 -- 172.21.15.145:0/2963953461 >> 172.21.15.145:0/2963953461 conn(0x558fd3398160 msgr2=0x558fd341e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 -- 172.21.15.145:0/2963953461 shutdown_connections 2023-08-25T04:44:55.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.146+0000 7f3baec02b80 1 -- 172.21.15.145:0/2963953461 wait complete. 2023-08-25T04:44:55.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.147+0000 7f3baec02b80 1 -- start start 2023-08-25T04:44:55.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.147+0000 7f3baec02b80 1 Processor -- start 2023-08-25T04:44:55.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd342d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3424dd0 0x558fd34349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd3375a10 con 0x558fd34243b0 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd3313c30 con 0x558fd3423a50 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd3313930 con 0x558fd3424dd0 2023-08-25T04:44:55.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3baec02b80 10 client.0 _ll_register_callbacks cb 0x558fd325db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd342d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46040/0 (socket says 172.21.15.145:46040) 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3b98f9b700 1 -- 172.21.15.145:0/2821655284 learned_addr learned my addr 172.21.15.145:0/2821655284 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.148+0000 7f3b9979c700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3424dd0 0x558fd34349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 -- 172.21.15.145:0/2821655284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3424dd0 msgr2=0x558fd34349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3424dd0 0x558fd34349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 -- 172.21.15.145:0/2821655284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 msgr2=0x558fd342d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 -- 172.21.15.145:0/2821655284 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b88009960 con 0x558fd34243b0 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b93fff700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd342d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b80096850 tx=0x7f3b8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cad437a33a668e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b73fff700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80099de0 con 0x558fd34243b0 2023-08-25T04:44:55.159 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3b8009dc10 con 0x558fd34243b0 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fd33f3240 con 0x558fd34243b0 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b800c9f10 con 0x558fd34243b0 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3baec02b80 10 client.15456 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.149+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558fd31a1e60 con 0x558fd34243b0 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b800b4030 con 0x558fd34243b0 2023-08-25T04:44:55.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b80005490 con 0x558fd34243b0 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 20 client.15456 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd3438ce0 0x558fd343b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 4 client.15456 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:55.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fd331bc20 con 0x558fd3438ce0 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.150+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd3438ce0 0x558fd343b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.151+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd3438ce0 0x558fd343b0d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0eae754984a82b6 server_cookie=948f643151baaebd in_seq=0 out_seq=0 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.151+0000 7f3b73fff700 10 client.15456 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.151+0000 7f3b73fff700 1 -- 172.21.15.145:0/2821655284 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fd331bc20 con 0x558fd3438ce0 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.151+0000 7f3b73fff700 10 client.15456 handle_command_reply: tid=0 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.151+0000 7f3baec02b80 1 client.15456 shutdown 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 msgr2=0x558fd342d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd342d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b80096850 tx=0x7f3b8000ee80 comp rx=0 tx=0).stop 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 shutdown_connections 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd3438ce0 0x558fd343b0d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3424dd0 0x558fd34349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3423a50 0x558fd342d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 --2- 172.21.15.145:0/2821655284 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd342d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 >> 172.21.15.145:0/2821655284 conn(0x558fd3398160 msgr2=0x558fd3418bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3b73fff700 0 client.15456 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 shutdown_connections 2023-08-25T04:44:55.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 1 -- 172.21.15.145:0/2821655284 wait complete. 2023-08-25T04:44:55.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.152+0000 7f3baec02b80 20 client.15456 trim_cache size 0 max 16384 2023-08-25T04:44:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:54 smithi162 ceph-mon[130941]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:44:55.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3363103503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:55.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:54 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1387016962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 Processor -- start 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 -- start start 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 0x558fd342b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3427cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd31f79a0 con 0x558fd3423a50 2023-08-25T04:44:55.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd341a890 con 0x558fd3424dd0 2023-08-25T04:44:55.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.170+0000 7f3baec02b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd342e0d0 con 0x558fd34243b0 2023-08-25T04:44:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:54 smithi183 ceph-mon[131414]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:44:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3363103503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T04:44:55.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:54 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1387016962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:44:55.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b98f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 0x558fd342b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:39690/0 (socket says 172.21.15.145:39690) 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 -- 172.21.15.145:0/4177490086 learned_addr learned my addr 172.21.15.145:0/4177490086 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b9979c700 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3427cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 -- 172.21.15.145:0/4177490086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 msgr2=0x558fd3427cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3427cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 -- 172.21.15.145:0/4177490086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 msgr2=0x558fd342b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 0x558fd342b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 -- 172.21.15.145:0/4177490086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd318ace0 con 0x558fd34243b0 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 0x558fd342b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.171+0000 7f3b93fff700 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005e90 tx=0x7f3b8000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e57323ae3eb2404 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3b7f7fe700 1 -- 172.21.15.145:0/4177490086 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80096670 con 0x558fd34243b0 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3b7f7fe700 1 -- 172.21.15.145:0/4177490086 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b800b4070 con 0x558fd34243b0 2023-08-25T04:44:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3b7f7fe700 1 -- 172.21.15.145:0/4177490086 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800055b0 con 0x558fd34243b0 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 -- 172.21.15.145:0/4177490086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 msgr2=0x558fd3434990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005e90 tx=0x7f3b8000ef70 comp rx=0 tx=0).stop 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 -- 172.21.15.145:0/4177490086 shutdown_connections 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3427cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd3423a50 0x558fd342b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 --2- 172.21.15.145:0/4177490086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd34243b0 0x558fd3434990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 -- 172.21.15.145:0/4177490086 >> 172.21.15.145:0/4177490086 conn(0x558fd3398160 msgr2=0x558fd341e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.172+0000 7f3baec02b80 1 -- 172.21.15.145:0/4177490086 shutdown_connections 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 -- 172.21.15.145:0/4177490086 wait complete. 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 -- start start 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 Processor -- start 2023-08-25T04:44:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3430600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3433b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd34340a0 con 0x558fd34243b0 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd3434210 con 0x558fd3424dd0 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.173+0000 7f3baec02b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd34343d0 con 0x558fd3423a50 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3baec02b80 10 client.0 _ll_register_callbacks cb 0x7f3b80099740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b9979c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3433b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57236/0 (socket says 172.21.15.145:57236) 2023-08-25T04:44:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 -- 172.21.15.145:0/711511153 learned_addr learned my addr 172.21.15.145:0/711511153 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b9979c700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3433b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46066/0 (socket says 172.21.15.145:46066) 2023-08-25T04:44:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3430600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 -- 172.21.15.145:0/711511153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 msgr2=0x558fd3430600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3430600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 -- 172.21.15.145:0/711511153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 msgr2=0x558fd3433b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3433b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 -- 172.21.15.145:0/711511153 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fd31f56b0 con 0x558fd34243b0 2023-08-25T04:44:55.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b93fff700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b80099ae0 tx=0x7f3b800051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0b078b760cee37 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.174+0000 7f3b73fff700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8009dc10 con 0x558fd34243b0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3b80099de0 con 0x558fd34243b0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fd3416930 con 0x558fd34243b0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b800b90b0 con 0x558fd34243b0 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 10 client.15462 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f3b88009960 con 0x558fd34243b0 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8007c070 con 0x558fd34243b0 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b8009a160 con 0x558fd34243b0 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 20 client.15462 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd348d480 0x558fd348f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 4 client.15462 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.175+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558fd336e9f0 con 0x558fd348d480 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.176+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd348d480 0x558fd348f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.176+0000 7f3b73fff700 10 client.15462 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.176+0000 7f3b98f9b700 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd348d480 0x558fd348f8a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2baa1fdcb0ef2b server_cookie=dc981ec71bbc6eef in_seq=0 out_seq=0 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3b73fff700 1 -- 172.21.15.145:0/711511153 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558fd336e9f0 con 0x558fd348d480 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3b73fff700 10 client.15462 handle_command_reply: tid=0 2023-08-25T04:44:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 client.15462 shutdown 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 msgr2=0x558fd3430b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b80099ae0 tx=0x7f3b800051b0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 shutdown_connections 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fd3424dd0 0x558fd3433b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fd348d480 0x558fd348f8a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fd3423a50 0x558fd3430600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 --2- 172.21.15.145:0/711511153 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fd34243b0 0x558fd3430b40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 >> 172.21.15.145:0/711511153 conn(0x558fd3398160 msgr2=0x558fd342b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3b73fff700 0 client.15462 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 shutdown_connections 2023-08-25T04:44:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.177+0000 7f3baec02b80 1 -- 172.21.15.145:0/711511153 wait complete. 2023-08-25T04:44:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3baec02b80 20 client.15462 trim_cache size 0 max 16384 2023-08-25T04:44:55.190 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:55.191 INFO:teuthology.orchestra.run.smithi145.stdout: "return_code": 0, 2023-08-25T04:44:55.191 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_tag": "9b059baa-5bea-4b6a-8463-237705658ba4", 2023-08-25T04:44:55.191 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": "asynchronous" 2023-08-25T04:44:55.191 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:55.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 msgr2=0x7f3b840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.192 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 0x7f3b840a2410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009ec0 tx=0x7f3b8c00b040 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 msgr2=0x7f3b94196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009230 tx=0x7f3b8000bf80 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 shutdown_connections 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f3b8409ff50 0x7f3b840a2410 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b941024a0 0x7f3b9419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100910 0x7f3b94196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 --2- 172.21.15.145:0/1455305687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b94101b00 0x7f3b94196ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.178+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 >> 172.21.15.145:0/1455305687 conn(0x7f3b940fbe60 msgr2=0x7f3b941050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.179+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 shutdown_connections 2023-08-25T04:44:55.194 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.179+0000 7f3b9b1ff700 1 -- 172.21.15.145:0/1455305687 wait complete. 2023-08-25T04:44:55.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:44:55.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 -- 172.21.15.145:0/3885005850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c073380 msgr2=0x7f627c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 --2- 172.21.15.145:0/3885005850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c073380 0x7f627c073800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62640098f0 tx=0x7f6264009c00 comp rx=0 tx=0).stop 2023-08-25T04:44:55.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 -- 172.21.15.145:0/3885005850 shutdown_connections 2023-08-25T04:44:55.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 --2- 172.21.15.145:0/3885005850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c1087f0 0x7f627c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 --2- 172.21.15.145:0/3885005850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 --2- 172.21.15.145:0/3885005850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c073380 0x7f627c073800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 -- 172.21.15.145:0/3885005850 >> 172.21.15.145:0/3885005850 conn(0x7f627c0fbea0 msgr2=0x7f627c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 -- 172.21.15.145:0/3885005850 shutdown_connections 2023-08-25T04:44:55.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.397+0000 7f6280bff700 1 -- 172.21.15.145:0/3885005850 wait complete. 2023-08-25T04:44:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.398+0000 7f6280bff700 1 Processor -- start 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 -- start start 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 0x7f627c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c1087f0 0x7f627c19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a0460 con 0x7f627c074960 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a05a0 con 0x7f627c1087f0 2023-08-25T04:44:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6280bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1a0730 con 0x7f627c073380 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f627a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 0x7f627c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57260/0 (socket says 172.21.15.145:57260) 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 -- 172.21.15.145:0/1614957024 learned_addr learned my addr 172.21.15.145:0/1614957024 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c1087f0 0x7f627c19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 -- 172.21.15.145:0/1614957024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 msgr2=0x7f627c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 0x7f627c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 -- 172.21.15.145:0/1614957024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c1087f0 msgr2=0x7f627c19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.399+0000 7f6273fff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c1087f0 0x7f627c19fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f6273fff700 1 -- 172.21.15.145:0/1614957024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0095f0 con 0x7f627c074960 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f627a59c700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 0x7f627c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f6273fff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6264009fd0 tx=0x7f626400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f62737fe700 1 -- 172.21.15.145:0/1614957024 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626401c070 con 0x7f627c074960 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f62737fe700 1 -- 172.21.15.145:0/1614957024 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626400f7d0 con 0x7f627c074960 2023-08-25T04:44:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6274009c40 con 0x7f627c074960 2023-08-25T04:44:55.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f62737fe700 1 -- 172.21.15.145:0/1614957024 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264003a20 con 0x7f627c074960 2023-08-25T04:44:55.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.400+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6264009520 con 0x7f627c074960 2023-08-25T04:44:55.408 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.404+0000 7f62737fe700 1 -- 172.21.15.145:0/1614957024 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 134672+0+0 (secure 0 0 0) 0x7f6264020030 con 0x7f627c074960 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.405+0000 7f6294592b80 1 Processor -- start 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.405+0000 7f62737fe700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 0x7f62680a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.405+0000 7f6294592b80 1 -- start start 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.405+0000 7f62737fe700 1 -- 172.21.15.145:0/1614957024 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f62640cc460 con 0x7f627c074960 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 0x7f62680a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 0x55e19c6cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6357b0 con 0x55e19c6ce330 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c626780 con 0x55e19c6cd9d0 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6294592b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6240a0 con 0x55e19c6ced50 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6273fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46092/0 (socket says 172.21.15.145:46092) 2023-08-25T04:44:55.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 -- 172.21.15.145:0/541182651 learned_addr learned my addr 172.21.15.145:0/541182651 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 -- 172.21.15.145:0/541182651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 msgr2=0x55e19c6cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627ad9d700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 0x55e19c6cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 0x55e19c6cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 -- 172.21.15.145:0/541182651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 msgr2=0x55e19c6ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627a59c700 1 -- 172.21.15.145:0/541182651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19c49f630 con 0x55e19c6cd9d0 2023-08-25T04:44:55.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f6273fff700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.406+0000 7f627ad9d700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 0x55e19c6cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.407+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 0x7f62680a2490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6274009fd0 tx=0x7f627400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.407+0000 7f627a59c700 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f626c0117e0 tx=0x7f626c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824f9c67680e3bfb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.407+0000 7f6270ff9700 1 -- 172.21.15.145:0/541182651 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c004020 con 0x55e19c6cd9d0 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.407+0000 7f6270ff9700 1 -- 172.21.15.145:0/541182651 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626c003680 con 0x55e19c6cd9d0 2023-08-25T04:44:55.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.407+0000 7f6270ff9700 1 -- 172.21.15.145:0/541182651 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00f840 con 0x55e19c6cd9d0 2023-08-25T04:44:55.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- 172.21.15.145:0/541182651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 msgr2=0x55e19c6cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f626c0117e0 tx=0x7f626c011af0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- 172.21.15.145:0/541182651 shutdown_connections 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ced50 0x55e19c6cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6cddf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 --2- 172.21.15.145:0/541182651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- 172.21.15.145:0/541182651 >> 172.21.15.145:0/541182651 conn(0x55e19c6420e0 msgr2=0x55e19c6c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- 172.21.15.145:0/541182651 shutdown_connections 2023-08-25T04:44:55.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- 172.21.15.145:0/541182651 wait complete. 2023-08-25T04:44:55.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.408+0000 7f6294592b80 1 -- start start 2023-08-25T04:44:55.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 Processor -- start 2023-08-25T04:44:55.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c61f990 con 0x55e19c6ced50 2023-08-25T04:44:55.420 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c5bdbb0 con 0x55e19c6cd9d0 2023-08-25T04:44:55.420 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.409+0000 7f6294592b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c5bd8b0 con 0x55e19c6ce330 2023-08-25T04:44:55.420 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f6294592b80 10 client.0 _ll_register_callbacks cb 0x55e19c507a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:55.420 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f6273fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f6273fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6d7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:39736/0 (socket says 172.21.15.145:39736) 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f6273fff700 1 -- 172.21.15.145:0/1807290653 learned_addr learned my addr 172.21.15.145:0/1807290653 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57282/0 (socket says 172.21.15.145:57282) 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 -- 172.21.15.145:0/1807290653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 msgr2=0x55e19c6d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 -- 172.21.15.145:0/1807290653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 msgr2=0x55e19c6d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6d73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.410+0000 7f627ad9d700 1 -- 172.21.15.145:0/1807290653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6264009520 con 0x55e19c6ced50 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6274017ed0 tx=0x7f6274004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a441b50e1aa735b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400e040 con 0x55e19c6ced50 2023-08-25T04:44:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e19c69d1c0 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6274009020 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62740091e0 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6294592b80 10 client.15474 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f626c011440 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.411+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e19c69d1c0 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6274004530 con 0x55e19c6ced50 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 20 client.15474 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c6e3120 0x55e19c6e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 4 client.15474 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e19c5c5ba0 con 0x55e19c6e3120 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f627a59c700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c6e3120 0x55e19c6e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f6261ffb700 10 client.15474 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.412+0000 7f627a59c700 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c6e3120 0x55e19c6e5540 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466c48dfc498bdf0 server_cookie=c81a3d0f44d9be5 in_seq=0 out_seq=0 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6261ffb700 1 -- 172.21.15.145:0/1807290653 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e19c5c5ba0 con 0x55e19c6e3120 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6261ffb700 10 client.15474 handle_command_reply: tid=0 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 client.15474 shutdown 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 msgr2=0x55e19c6de950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6274017ed0 tx=0x7f6274004ad0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 shutdown_connections 2023-08-25T04:44:55.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6de950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.413+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c6e3120 0x55e19c6e5540 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6294592b80 1 --2- 172.21.15.145:0/1807290653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 >> 172.21.15.145:0/1807290653 conn(0x55e19c6420e0 msgr2=0x55e19c6c2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6261ffb700 0 client.15474 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 shutdown_connections 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6294592b80 1 -- 172.21.15.145:0/1807290653 wait complete. 2023-08-25T04:44:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.414+0000 7f6294592b80 20 client.15474 trim_cache size 0 max 16384 2023-08-25T04:44:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.434+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f10880ffbb0 con 0x7f10881024e0 2023-08-25T04:44:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.441+0000 7f6294592b80 1 Processor -- start 2023-08-25T04:44:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.441+0000 7f6294592b80 1 -- start start 2023-08-25T04:44:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c4a1920 con 0x55e19c6ced50 2023-08-25T04:44:55.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6d6f40 con 0x55e19c6cd9d0 2023-08-25T04:44:55.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6294592b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6c9590 con 0x55e19c6ce330 2023-08-25T04:44:55.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f627ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f627ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57284/0 (socket says 172.21.15.145:57284) 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f627ad9d700 1 -- 172.21.15.145:0/1297260038 learned_addr learned my addr 172.21.15.145:0/1297260038 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6273fff700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f627a59c700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6273fff700 1 -- 172.21.15.145:0/1297260038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 msgr2=0x55e19c6c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6273fff700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6273fff700 1 -- 172.21.15.145:0/1297260038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 msgr2=0x55e19c6df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.442+0000 7f6273fff700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f6273fff700 1 -- 172.21.15.145:0/1297260038 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19c44bde0 con 0x55e19c6ce330 2023-08-25T04:44:55.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f6273fff700 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6de870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6264099b70 tx=0x7f6264019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e084db7f62bd2825 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f62717fa700 1 -- 172.21.15.145:0/1297260038 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640cc040 con 0x55e19c6ce330 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f62717fa700 1 -- 172.21.15.145:0/1297260038 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626409a2e0 con 0x55e19c6ce330 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.443+0000 7f62717fa700 1 -- 172.21.15.145:0/1297260038 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264097f80 con 0x55e19c6ce330 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- 172.21.15.145:0/1297260038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 msgr2=0x55e19c6de870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6de870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6264099b70 tx=0x7f6264019040 comp rx=0 tx=0).stop 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- 172.21.15.145:0/1297260038 shutdown_connections 2023-08-25T04:44:55.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ced50 0x55e19c6df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6cd9d0 0x55e19c6c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 --2- 172.21.15.145:0/1297260038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6ce330 0x55e19c6de870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- 172.21.15.145:0/1297260038 >> 172.21.15.145:0/1297260038 conn(0x55e19c6420e0 msgr2=0x55e19c6c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- 172.21.15.145:0/1297260038 shutdown_connections 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- 172.21.15.145:0/1297260038 wait complete. 2023-08-25T04:44:55.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.444+0000 7f6294592b80 1 -- start start 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 Processor -- start 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 0x55e19c6da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6ced50 0x55e19c6ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6de0c0 con 0x55e19c6ce330 2023-08-25T04:44:55.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6de230 con 0x55e19c6ced50 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c6de3f0 con 0x55e19c6cd9d0 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6294592b80 10 client.0 _ll_register_callbacks cb 0x7f62740178a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6273fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f627a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 0x55e19c6da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6273fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57292/0 (socket says 172.21.15.145:57292) 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.445+0000 7f6273fff700 1 -- 172.21.15.145:0/1395115361 learned_addr learned my addr 172.21.15.145:0/1395115361 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:55.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627ad9d700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6ced50 0x55e19c6ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 -- 172.21.15.145:0/1395115361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6ced50 msgr2=0x55e19c6ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6ced50 0x55e19c6ddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 -- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 msgr2=0x55e19c6dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 -- 172.21.15.145:0/1395115361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c011440 con 0x55e19c6cd9d0 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f6273fff700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f627a59c700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 0x55e19c6da620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9b860c0e8e9d2b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:55.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f6261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c02a040 con 0x55e19c6cd9d0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e19c6c08b0 con 0x55e19c6cd9d0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.446+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f626c003680 con 0x55e19c6cd9d0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f626c003f90 con 0x55e19c6cd9d0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 10 client.25489 fetch_fsmap learned FSMap version 23 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f6274009c40 con 0x55e19c6cd9d0 2023-08-25T04:44:55.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e19c6c08b0 con 0x55e19c6cd9d0 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f626c02d070 con 0x55e19c6cd9d0 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 10 client.25489 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 20 client.25489 populate_metadata read hostname 'smithi145' 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c737f50 0x55e19c73a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 4 client.25489 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.447+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e19c618970 con 0x55e19c737f50 2023-08-25T04:44:55.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6273fff700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c737f50 0x55e19c73a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6261ffb700 10 client.25489 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6273fff700 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c737f50 0x55e19c73a370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=defe6a7da60cb7a6 server_cookie=e82a4ba740f4138d in_seq=0 out_seq=0 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6261ffb700 1 -- 172.21.15.145:0/1395115361 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e19c618970 con 0x55e19c737f50 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6261ffb700 10 client.25489 handle_command_reply: tid=0 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 client.25489 shutdown 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 msgr2=0x55e19c6da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 0x55e19c6da620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c009f30 comp rx=0 tx=0).stop 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 shutdown_connections 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e19c6ced50 0x55e19c6ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e19c737f50 0x55e19c73a370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e19c6cd9d0 0x55e19c6da620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 --2- 172.21.15.145:0/1395115361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e19c6ce330 0x55e19c6dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 >> 172.21.15.145:0/1395115361 conn(0x55e19c6420e0 msgr2=0x55e19c6c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.448+0000 7f6261ffb700 0 client.25489 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:44:55.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 shutdown_connections 2023-08-25T04:44:55.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6294592b80 1 -- 172.21.15.145:0/1395115361 wait complete. 2023-08-25T04:44:55.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6294592b80 20 client.25489 trim_cache size 0 max 16384 2023-08-25T04:44:55.459 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:44:55.459 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "IDLE (waiting for more scrubs)", 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "9b059baa-5bea-4b6a-8463-237705658ba4": { 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "9b059baa-5bea-4b6a-8463-237705658ba4", 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:44:55.460 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:44:55.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 msgr2=0x7f62680a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 0x7f62680a2490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6274009fd0 tx=0x7f627400b040 comp rx=0 tx=0).stop 2023-08-25T04:44:55.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 msgr2=0x7f627c198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6264009fd0 tx=0x7f626400b790 comp rx=0 tx=0).stop 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 shutdown_connections 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f626809ffd0 0x7f62680a2490 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f627c1087f0 0x7f627c19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f627c073380 0x7f627c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 --2- 172.21.15.145:0/1614957024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f627c074960 0x7f627c198e20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.449+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 >> 172.21.15.145:0/1614957024 conn(0x7f627c0fbea0 msgr2=0x7f627c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.450+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 shutdown_connections 2023-08-25T04:44:55.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.450+0000 7f6280bff700 1 -- 172.21.15.145:0/1614957024 wait complete. 2023-08-25T04:44:55.468 INFO:tasks.cephfs.filesystem:scrub status for tag:9b059baa-5bea-4b6a-8463-237705658ba4 - {'path': '/', 'tag': '9b059baa-5bea-4b6a-8463-237705658ba4', 'options': 'recursive,force'} 2023-08-25T04:44:55.841 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.839+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 17) v1 ==== 134689+0+0 (secure 0 0 0) 0x7f1080003cb0 con 0x7f10881024e0 2023-08-25T04:44:55.842 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.840+0000 7f1085ffb700 1 -- 172.21.15.145:0/75999066 <== mon.0 v2:172.21.15.145:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v17) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1080098620 con 0x7f10881024e0 2023-08-25T04:44:55.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 msgr2=0x7f10780a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 0x7f10780a2410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f107c009990 tx=0x7f107c005fb0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 msgr2=0x7f1088199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:55.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f108000ef30 tx=0x7f108000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 shutdown_connections 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f107809ff50 0x7f10780a2410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10881024e0 0x7f1088199ce0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1088100950 0x7f1088198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 --2- 172.21.15.145:0/75999066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1088101b40 0x7f1088198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 >> 172.21.15.145:0/75999066 conn(0x7f10880fbea0 msgr2=0x7f10880fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 shutdown_connections 2023-08-25T04:44:55.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:55.844+0000 7f108efff700 1 -- 172.21.15.145:0/75999066 wait complete. 2023-08-25T04:44:55.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-08-25T04:44:56.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 -- 172.21.15.145:0/791551506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c074960 msgr2=0x7f503c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 --2- 172.21.15.145:0/791551506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c074960 0x7f503c072db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f502c009990 tx=0x7f502c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 -- 172.21.15.145:0/791551506 shutdown_connections 2023-08-25T04:44:56.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 --2- 172.21.15.145:0/791551506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 0x7f503c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 --2- 172.21.15.145:0/791551506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c074960 0x7f503c072db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 --2- 172.21.15.145:0/791551506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c073380 0x7f503c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 -- 172.21.15.145:0/791551506 >> 172.21.15.145:0/791551506 conn(0x7f503c0fbea0 msgr2=0x7f503c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:56.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 -- 172.21.15.145:0/791551506 shutdown_connections 2023-08-25T04:44:56.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.069+0000 7f50425ff700 1 -- 172.21.15.145:0/791551506 wait complete. 2023-08-25T04:44:56.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.070+0000 7f50425ff700 1 Processor -- start 2023-08-25T04:44:56.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.070+0000 7f50425ff700 1 -- start start 2023-08-25T04:44:56.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 0x7f503c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 0x7f503c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c19a360 con 0x7f503c074960 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c19a4d0 con 0x7f503c073380 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f50425ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c19a690 con 0x7f503c108810 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 0x7f503c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46148/0 (socket says 172.21.15.145:46148) 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 -- 172.21.15.145:0/1985661915 learned_addr learned my addr 172.21.15.145:0/1985661915 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f5040b9c700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 0x7f503c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 msgr2=0x7f503c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 0x7f503c199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 msgr2=0x7f503c198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 0x7f503c198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503bfff700 1 -- 172.21.15.145:0/1985661915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0095f0 con 0x7f503c073380 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.071+0000 7f503b7fe700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 0x7f503c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f503bfff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f502c004740 tx=0x7f502c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c01d070 con 0x7f503c073380 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c004830 con 0x7f503c073380 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0053c0 con 0x7f503c073380 2023-08-25T04:44:56.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c19a910 con 0x7f503c073380 2023-08-25T04:44:56.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.072+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c1a0f50 con 0x7f503c073380 2023-08-25T04:44:56.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.073+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c106ec0 con 0x7f503c073380 2023-08-25T04:44:56.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.075+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 134689+0+0 (secure 0 0 0) 0x7f502c005580 con 0x7f503c073380 2023-08-25T04:44:56.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.076+0000 7f50397fa700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 0x7f50240a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.076+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f502c0cc760 con 0x7f503c073380 2023-08-25T04:44:56.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.076+0000 7f503b7fe700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 msgr2=0x7f50240a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:56.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.076+0000 7f503b7fe700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 0x7f50240a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:44:56.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.078+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f502c096220 con 0x7f503c073380 2023-08-25T04:44:56.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2636145651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:56.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: scrub queued for path: / 2023-08-25T04:44:56.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:44:56.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.093 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T04:44:56.093 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: ignoring --setuser ceph since I am not root 2023-08-25T04:44:56.093 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: ignoring --setgroup ceph since I am not root 2023-08-25T04:44:56.093 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:55 smithi183 ceph-mgr[133323]: -- 172.21.15.183:0/110262974 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602b19ad1e0 con 0x5602b1982c00 2023-08-25T04:44:56.093 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.011+0000 7f0514d58200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:44:56.094 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.091+0000 7f0514d58200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:44:56.094 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2636145651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:56.094 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: scrub queued for path: / 2023-08-25T04:44:56.094 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.094 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: ignoring --setuser ceph since I am not root 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: ignoring --setgroup ceph since I am not root 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:55 smithi162 ceph-mgr[132416]: -- 172.21.15.162:0/800462060 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a6a9971e0 con 0x564a6a96cc00 2023-08-25T04:44:56.095 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.011+0000 7f0b29fe9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:44:56.119 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setuser ceph since I am not root 2023-08-25T04:44:56.119 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: ignoring --setgroup ceph since I am not root 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2636145651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: scrub queued for path: / 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:44:56.120 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:55 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-08-25T04:44:56.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.277+0000 7f503b7fe700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 msgr2=0x7f50240a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:56.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.277+0000 7f503b7fe700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 0x7f50240a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:44:56.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.288+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f503c04e700 con 0x7f503c073380 2023-08-25T04:44:56.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.295+0000 7f50397fa700 1 -- 172.21.15.145:0/1985661915 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f502c099840 con 0x7f503c073380 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 msgr2=0x7f50240a2460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 0x7f50240a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 msgr2=0x7f503c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f502c004740 tx=0x7f502c009580 comp rx=0 tx=0).stop 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 shutdown_connections 2023-08-25T04:44:56.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f502409ffa0 0x7f50240a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f503c108810 0x7f503c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503c073380 0x7f503c198940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 --2- 172.21.15.145:0/1985661915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f503c074960 0x7f503c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 >> 172.21.15.145:0/1985661915 conn(0x7f503c0fbea0 msgr2=0x7f503c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 shutdown_connections 2023-08-25T04:44:56.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.298+0000 7f50425ff700 1 -- 172.21.15.145:0/1985661915 wait complete. 2023-08-25T04:44:56.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-08-25T04:44:56.380 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:56.117+0000 7f0eb687a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T04:44:56.380 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:56.224+0000 7f0eb687a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:44:56.380 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:56.349+0000 7f0eb687a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:44:56.411 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.092+0000 7f0b29fe9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T04:44:56.411 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.184+0000 7f0b29fe9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:44:56.412 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.277+0000 7f0b29fe9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:44:56.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.181+0000 7f0514d58200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T04:44:56.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.275+0000 7f0514d58200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:44:56.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 -- 172.21.15.145:0/4222232821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 msgr2=0x7f57f4108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 --2- 172.21.15.145:0/4222232821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4108d70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b0d0 tx=0x7f57ec00b3e0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 -- 172.21.15.145:0/4222232821 shutdown_connections 2023-08-25T04:44:56.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 --2- 172.21.15.145:0/4222232821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4108d70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 --2- 172.21.15.145:0/4222232821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 --2- 172.21.15.145:0/4222232821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 -- 172.21.15.145:0/4222232821 >> 172.21.15.145:0/4222232821 conn(0x7f57f40fbe60 msgr2=0x7f57f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:56.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 -- 172.21.15.145:0/4222232821 shutdown_connections 2023-08-25T04:44:56.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.518+0000 7f57f97ff700 1 -- 172.21.15.145:0/4222232821 wait complete. 2023-08-25T04:44:56.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 Processor -- start 2023-08-25T04:44:56.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 -- start start 2023-08-25T04:44:56.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.520+0000 7f57f97ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f419a400 con 0x7f57f41024c0 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f97ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f419a570 con 0x7f57f4101b20 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f97ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f419a730 con 0x7f57f4100930 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:39812/0 (socket says 172.21.15.145:39812) 2023-08-25T04:44:56.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f2ffd700 1 -- 172.21.15.145:0/3876018590 learned_addr learned my addr 172.21.15.145:0/3876018590 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:56.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f37fe700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 msgr2=0x7f57f41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 msgr2=0x7f57f4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 -- 172.21.15.145:0/3876018590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009d00 con 0x7f57f4101b20 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f2ffd700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f37fe700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.521+0000 7f57f27fc700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4198f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d750 tx=0x7f57e800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.522+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8004020 con 0x7f57f4101b20 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.522+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57e8004830 con 0x7f57f4101b20 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.522+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80054c0 con 0x7f57f4101b20 2023-08-25T04:44:56.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.522+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f419aa10 con 0x7f57f4101b20 2023-08-25T04:44:56.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.522+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f41a11d0 con 0x7f57f4101b20 2023-08-25T04:44:56.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.523+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f404e700 con 0x7f57f4101b20 2023-08-25T04:44:56.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.524+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 134689+0+0 (secure 0 0 0) 0x7f57e80049a0 con 0x7f57f4101b20 2023-08-25T04:44:56.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.526+0000 7f57dbfff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 0x7f57dc0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.526+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f57e80cae40 con 0x7f57f4101b20 2023-08-25T04:44:56.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.526+0000 7f57f2ffd700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 msgr2=0x7f57dc0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:56.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.526+0000 7f57f2ffd700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 0x7f57dc0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:44:56.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.530+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f57e8094870 con 0x7f57f4101b20 2023-08-25T04:44:56.713 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:56.477+0000 7f0eb687a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T04:44:56.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.727+0000 7f57f2ffd700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 msgr2=0x7f57dc0a24e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:56.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.727+0000 7f57f2ffd700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 0x7f57dc0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:44:56.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.737+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f57f410c150 con 0x7f57f4101b20 2023-08-25T04:44:56.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.743+0000 7f57dbfff700 1 -- 172.21.15.145:0/3876018590 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f57e801d0c0 con 0x7f57f4101b20 2023-08-25T04:44:56.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 msgr2=0x7f57dc0a24e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 0x7f57dc0a24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 msgr2=0x7f57f4198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4198f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d750 tx=0x7f57e800da60 comp rx=0 tx=0).stop 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 shutdown_connections 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7f57dc0a0020 0x7f57dc0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57f41024c0 0x7f57f4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57f4100930 0x7f57f41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 --2- 172.21.15.145:0/3876018590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57f4101b20 0x7f57f4198f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 >> 172.21.15.145:0/3876018590 conn(0x7f57f40fbe60 msgr2=0x7f57f41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:56.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 shutdown_connections 2023-08-25T04:44:56.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.746+0000 7f57f97ff700 1 -- 172.21.15.145:0/3876018590 wait complete. 2023-08-25T04:44:56.767 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-08-25T04:44:56.808 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.446+0000 7f0514d58200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:44:56.813 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.448+0000 7f0b29fe9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:44:56.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 -- 172.21.15.145:0/3658776046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 msgr2=0x7fa700073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 --2- 172.21.15.145:0/3658776046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa700073800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009990 tx=0x7fa6f0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 -- 172.21.15.145:0/3658776046 shutdown_connections 2023-08-25T04:44:56.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 --2- 172.21.15.145:0/3658776046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 0x7fa70010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 --2- 172.21.15.145:0/3658776046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 0x7fa700072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 --2- 172.21.15.145:0/3658776046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa700073800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 -- 172.21.15.145:0/3658776046 >> 172.21.15.145:0/3658776046 conn(0x7fa7000fbe60 msgr2=0x7fa7000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:44:56.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 -- 172.21.15.145:0/3658776046 shutdown_connections 2023-08-25T04:44:56.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.965+0000 7fa707dff700 1 -- 172.21.15.145:0/3658776046 wait complete. 2023-08-25T04:44:56.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 Processor -- start 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 -- start start 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa70019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 0x7fa70019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 0x7fa70019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70019e7c0 con 0x7fa70010cf00 2023-08-25T04:44:56.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70019e930 con 0x7fa700073380 2023-08-25T04:44:56.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.966+0000 7fa707dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70019eaf0 con 0x7fa700074960 2023-08-25T04:44:56.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa705b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa70019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 0x7fa70019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa705b9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa70019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46186/0 (socket says 172.21.15.145:46186) 2023-08-25T04:44:56.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 learned_addr learned my addr 172.21.15.145:0/2169035472 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:44:56.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70639c700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 0x7fa70019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:44:56.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 msgr2=0x7fa70019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa700073380 0x7fa70019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 msgr2=0x7fa70019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:44:56.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 0x7fa70019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:44:56.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00095f0 con 0x7fa700074960 2023-08-25T04:44:56.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70639c700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa70010cf00 0x7fa70019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:44:56.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa70539a700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 0x7fa70019d310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00098e0 tx=0x7fa6f0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:44:56.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f001d070 con 0x7fa700074960 2023-08-25T04:44:56.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6f0021410 con 0x7fa700074960 2023-08-25T04:44:56.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70019ed70 con 0x7fa700074960 2023-08-25T04:44:56.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.967+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0017650 con 0x7fa700074960 2023-08-25T04:44:56.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.968+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7001a5510 con 0x7fa700074960 2023-08-25T04:44:56.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.969+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa700102a50 con 0x7fa700074960 2023-08-25T04:44:56.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.970+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) v1 ==== 134689+0+0 (secure 0 0 0) 0x7fa6f0021580 con 0x7fa700074960 2023-08-25T04:44:56.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.972+0000 7fa6f6ffd700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:44:56.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.972+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:56.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.972+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa6f00cc9e0 con 0x7fa700074960 2023-08-25T04:44:56.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.972+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T04:44:56.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:56.974+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fa6f0096530 con 0x7fa700074960 2023-08-25T04:44:57.078 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:56.712+0000 7f0eb687a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T04:44:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T04:44:57.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-mon[97681]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-08-25T04:44:57.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:44:56 smithi145 ceph-mon[97681]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.123 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.808+0000 7f0514d58200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:44:57.124 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:56.939+0000 7f0514d58200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:44:57.124 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:57.028+0000 7f0514d58200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:44:57.124 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:57.123+0000 7f0514d58200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:44:57.124 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T04:44:57.124 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-mon[131414]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-08-25T04:44:57.125 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.125 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:44:56 smithi183 ceph-mon[131414]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.130 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.812+0000 7f0b29fe9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:44:57.130 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:56.944+0000 7f0b29fe9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:44:57.131 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:57.034+0000 7f0b29fe9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:44:57.131 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/75999066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-08-25T04:44:57.131 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-mon[130941]: mgrmap e17: x(active, since 8m), standbys: z, y 2023-08-25T04:44:57.131 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.131 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:44:56 smithi162 ceph-mon[130941]: from='client.? ' entity='client.admin' 2023-08-25T04:44:57.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:57.173+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:57.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:57.173+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T04:44:57.224 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:57.223+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa700060d60 con 0x7fa6ec09ffa0 2023-08-25T04:44:57.411 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:57.129+0000 7f0b29fe9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:44:57.481 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:57 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:57.197+0000 7f0eb687a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T04:44:57.482 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:57 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:57.362+0000 7f0eb687a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T04:44:57.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:57.574+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:57.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:57.574+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-25T04:44:57.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:57 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:57.480+0000 7f0eb687a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T04:44:57.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:57 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:57.604+0000 7f0eb687a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T04:44:57.913 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:57.551+0000 7f0b29fe9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:44:57.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:57.544+0000 7f0514d58200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:44:58.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:58.375+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:58.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:58.375+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-25T04:44:58.577 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:58.140+0000 7f0eb687a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T04:44:58.661 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:58.209+0000 7f0b29fe9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:44:58.662 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:58.292+0000 7f0b29fe9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:44:58.672 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:58.198+0000 7f0514d58200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:44:58.673 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:58.280+0000 7f0514d58200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:44:59.315 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.053+0000 7f0514d58200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:44:59.316 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.143+0000 7f0514d58200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:44:59.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:58 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:58.987+0000 7f0eb687a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T04:44:59.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:44:59 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:44:59.095+0000 7f0eb687a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T04:44:59.334 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.071+0000 7f0b29fe9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:44:59.334 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.161+0000 7f0b29fe9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:44:59.594 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.315+0000 7f0514d58200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:44:59.614 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.333+0000 7f0b29fe9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:44:59.911 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.613+0000 7f0b29fe9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:44:59.911 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.843+0000 7f0b29fe9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:44:59.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.594+0000 7f0514d58200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:44:59.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.822+0000 7f0514d58200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:44:59.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:59.977+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:44:59.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:44:59.977+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-25T04:45:00.233 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:44:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:44:59.970+0000 7f0514d58200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:45:00.255 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:44:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:44:59.991+0000 7f0b29fe9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:45:00.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:00.053+0000 7f0eb687a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:45:00.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:00.172+0000 7f0eb687a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T04:45:00.661 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:00.254+0000 7f0b29fe9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:45:00.672 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:00.233+0000 7f0514d58200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:45:00.759 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:00.396+0000 7f0eb687a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T04:45:01.036 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:00.758+0000 7f0eb687a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T04:45:01.254 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:00.932+0000 7f0514d58200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:45:01.255 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:01.062+0000 7f0514d58200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:45:01.282 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:00.957+0000 7f0b29fe9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:45:01.282 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:01.089+0000 7f0b29fe9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:45:01.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:01 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:01.036+0000 7f0eb687a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T04:45:01.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:01 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:01.227+0000 7f0eb687a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T04:45:01.584 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:01.254+0000 7f0514d58200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:45:01.584 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:01.367+0000 7f0514d58200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:45:01.611 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:01.281+0000 7f0b29fe9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:45:01.611 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:01.395+0000 7f0b29fe9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:45:01.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:01 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:01.571+0000 7f0eb687a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T04:45:01.911 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:01.610+0000 7f0b29fe9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:45:01.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:01.583+0000 7f0514d58200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:45:02.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:02.523+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 18) v1 ==== 134689+0+0 (secure 0 0 0) 0x7fa6f0096900 con 0x7fa700074960 2023-08-25T04:45:02.827 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:02.508+0000 7f0eb687a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T04:45:02.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:02.689+0000 7f0eb687a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T04:45:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: Standby manager daemon z restarted 2023-08-25T04:45:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: Standby manager daemon z started 2023-08-25T04:45:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:45:02.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: Standby manager daemon y restarted 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: Standby manager daemon y started 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:45:02.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:45:02.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-mon[97681]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:02.911 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y[132392]: 2023-08-25T04:45:02.479+0000 7f0b29fe9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:45:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: Standby manager daemon z restarted 2023-08-25T04:45:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: Standby manager daemon z started 2023-08-25T04:45:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:45:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: Standby manager daemon y restarted 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: Standby manager daemon y started 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:45:02.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:02 smithi162 ceph-mon[130941]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:02.922 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z[133280]: 2023-08-25T04:45:02.446+0000 7f0514d58200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:45:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: Standby manager daemon z restarted 2023-08-25T04:45:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: Standby manager daemon z started 2023-08-25T04:45:02.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-25T04:45:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-25T04:45:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.183:0/1365527376' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: Standby manager daemon y restarted 2023-08-25T04:45:02.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: Standby manager daemon y started 2023-08-25T04:45:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-25T04:45:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-25T04:45:02.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-25T04:45:02.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:02 smithi183 ceph-mon[131414]: from='mgr.? 172.21.15.162:0/943908131' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-25T04:45:03.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:03.180+0000 7fa705b9b700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.145:6800/891678096 2023-08-25T04:45:03.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:03.180+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-25T04:45:03.327 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:02 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:02.940+0000 7f0eb687a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T04:45:03.328 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:03.087+0000 7f0eb687a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T04:45:03.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:03 smithi145 ceph-mon[97681]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-08-25T04:45:03.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:03 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:03.366+0000 7f0eb687a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T04:45:03.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:03 smithi162 ceph-mon[130941]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-08-25T04:45:03.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:03 smithi183 ceph-mon[131414]: mgrmap e18: x(active, since 8m), standbys: y, z 2023-08-25T04:45:04.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:04.463+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 8 ==== mgrmap(e 19) v1 ==== 134356+0+0 (secure 0 0 0) 0x7fa6f00218a0 con 0x7fa700074960 2023-08-25T04:45:04.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:04.463+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 msgr2=0x7fa6ec0a2460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T04:45:04.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:04.463+0000 7fa6f6ffd700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/891678096,v1:172.21.15.145:6801/891678096] conn(0x7fa6ec09ffa0 0x7fa6ec0a2460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:04.767 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.444+0000 7f0eb687a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T04:45:04.767 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.659+0000 7f0e18aab700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.767 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.659+0000 7f0e18aab700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.659+0000 7f0e18aab700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.659+0000 7f0e18aab700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.659+0000 7f0e18aab700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.661+0000 7f0e16aa7700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.661+0000 7f0e16aa7700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.661+0000 7f0e16aa7700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.768 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.661+0000 7f0e16aa7700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.769 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:45:04.661+0000 7f0e16aa7700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:45:04.769 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: Active manager daemon x restarted 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: Activating manager daemon x 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: mgrmap e19: x(active, starting, since 0.0140292s), standbys: y, z 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:45:04.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:45:04.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:45:04.772 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:45:04.773 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:45:04.774 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:45:04.774 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:45:04.774 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:45:04.774 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:45:04.775 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:45:04.775 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: Manager daemon x is now available 2023-08-25T04:45:04.775 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:04.775 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:04 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: Active manager daemon x restarted 2023-08-25T04:45:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: Activating manager daemon x 2023-08-25T04:45:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T04:45:04.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: mgrmap e19: x(active, starting, since 0.0140292s), standbys: y, z 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:45:04.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:45:04.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:45:04.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:45:04.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:45:04.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:45:04.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:45:04.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:45:04.917 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:45:04.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: Manager daemon x is now available 2023-08-25T04:45:04.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:04.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:04 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:04.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: Active manager daemon x restarted 2023-08-25T04:45:04.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: Activating manager daemon x 2023-08-25T04:45:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: osdmap e90: 12 total, 12 up, 12 in 2023-08-25T04:45:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: mgrmap e19: x(active, starting, since 0.0140292s), standbys: y, z 2023-08-25T04:45:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-25T04:45:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-25T04:45:04.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-25T04:45:04.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-25T04:45:04.925 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-25T04:45:04.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-25T04:45:04.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-25T04:45:04.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-25T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-25T04:45:04.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: Manager daemon x is now available 2023-08-25T04:45:04.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:04.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:04 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:05.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.466+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mon.1 v2:172.21.15.183:3300/0 9 ==== mgrmap(e 20) v1 ==== 134483+0+0 (secure 0 0 0) 0x7fa6f00b91f0 con 0x7fa700074960 2023-08-25T04:45:05.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.467+0000 7fa6f6ffd700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa6ec034120 0x7fa6ec034500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:05.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.467+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa700060d60 con 0x7fa6ec034120 2023-08-25T04:45:05.470 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.470+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa6ec034120 0x7fa6ec034500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:05.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.474+0000 7fa705b9b700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa6ec034120 0x7fa6ec034500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa70019db70 tx=0x7fa6fc009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:05.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.534+0000 7fa6f6ffd700 1 -- 172.21.15.145:0/2169035472 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa700060d60 con 0x7fa6ec034120 2023-08-25T04:45:05.536 INFO:teuthology.orchestra.run.smithi145.stdout:Schedule set for path / 2023-08-25T04:45:05.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa6ec034120 msgr2=0x7fa6ec034500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:05.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa6ec034120 0x7fa6ec034500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa70019db70 tx=0x7fa6fc009260 comp rx=0 tx=0).stop 2023-08-25T04:45:05.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 msgr2=0x7fa70019d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 --2- 172.21.15.145:0/2169035472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa700074960 0x7fa70019d310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00098e0 tx=0x7fa6f0009580 comp rx=0 tx=0).stop 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa70639c700 1 -- 172.21.15.145:0/2169035472 reap_dead start 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 shutdown_connections 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 >> 172.21.15.145:0/2169035472 conn(0x7fa7000fbe60 msgr2=0x7fa7000fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 shutdown_connections 2023-08-25T04:45:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.536+0000 7fa707dff700 1 -- 172.21.15.145:0/2169035472 wait complete. 2023-08-25T04:45:05.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-08-25T04:45:05.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.761+0000 7fe7e43ff700 1 -- 172.21.15.145:0/1074800956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc1024e0 msgr2=0x7fe7dc108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:05.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.761+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/1074800956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc108d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b0a0 tx=0x7fe7d400b3b0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 -- 172.21.15.145:0/1074800956 shutdown_connections 2023-08-25T04:45:05.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/1074800956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc108d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/1074800956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc100950 0x7fe7dc100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/1074800956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 -- 172.21.15.145:0/1074800956 >> 172.21.15.145:0/1074800956 conn(0x7fe7dc0fbea0 msgr2=0x7fe7dc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:05.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 -- 172.21.15.145:0/1074800956 shutdown_connections 2023-08-25T04:45:05.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.762+0000 7fe7e43ff700 1 -- 172.21.15.145:0/1074800956 wait complete. 2023-08-25T04:45:05.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 Processor -- start 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 -- start start 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 0x7fe7dc072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a0760 con 0x7fe7dc101b40 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a08a0 con 0x7fe7dc1024e0 2023-08-25T04:45:05.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e43ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1a09e0 con 0x7fe7dc100950 2023-08-25T04:45:05.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e199a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:05.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e199a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58690/0 (socket says 172.21.15.145:58690) 2023-08-25T04:45:05.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e199a700 1 -- 172.21.15.145:0/2309479553 learned_addr learned my addr 172.21.15.145:0/2309479553 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:05.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.763+0000 7fe7e219b700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 0x7fe7dc072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e299c700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 -- 172.21.15.145:0/2309479553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 msgr2=0x7fe7dc072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 0x7fe7dc072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 -- 172.21.15.145:0/2309479553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc1024e0 msgr2=0x7fe7dc073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 -- 172.21.15.145:0/2309479553 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4009d00 con 0x7fe7dc101b40 2023-08-25T04:45:05.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e219b700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 0x7fe7dc072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e199a700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9a0 tx=0x7fe7cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc009790 con 0x7fe7dc101b40 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7cc010430 con 0x7fe7dc101b40 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc1a0c70 con 0x7fe7dc101b40 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc018680 con 0x7fe7dc101b40 2023-08-25T04:45:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.764+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc1a1180 con 0x7fe7dc101b40 2023-08-25T04:45:05.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.766+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc107ea0 con 0x7fe7dc101b40 2023-08-25T04:45:05.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.767+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 20) v1 ==== 134483+0+0 (secure 0 0 0) 0x7fe7cc0105a0 con 0x7fe7dc101b40 2023-08-25T04:45:05.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.769+0000 7fe7d37fe700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 0x7fe7c80a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:05.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.769+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe7cc014070 con 0x7fe7dc101b40 2023-08-25T04:45:05.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.769+0000 7fe7e219b700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 0x7fe7c80a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:05.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.770+0000 7fe7e219b700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 0x7fe7c80a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc074620 tx=0x7fe7d8009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:05.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:05.774+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe7cc051e00 con 0x7fe7dc101b40 2023-08-25T04:45:05.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:05 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:45:05.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:05 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:45:05.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:05 smithi145 ceph-mon[97681]: mgrmap e20: x(active, since 1.01849s), standbys: y, z 2023-08-25T04:45:05.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:05 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:45:05.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:05 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:45:05.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:05 smithi162 ceph-mon[130941]: mgrmap e20: x(active, since 1.01849s), standbys: y, z 2023-08-25T04:45:05.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:05 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-25T04:45:05.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:05 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-25T04:45:05.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:05 smithi183 ceph-mon[131414]: mgrmap e20: x(active, since 1.01849s), standbys: y, z 2023-08-25T04:45:06.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.021+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7dc02cbe0 con 0x7fe7c809fd50 2023-08-25T04:45:06.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.074+0000 7fe7d37fe700 1 -- 172.21.15.145:0/2309479553 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe7dc02cbe0 con 0x7fe7c809fd50 2023-08-25T04:45:06.075 INFO:teuthology.orchestra.run.smithi145.stdout:Retention added to path / 2023-08-25T04:45:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 msgr2=0x7fe7c80a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 0x7fe7c80a2210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc074620 tx=0x7fe7d8009260 comp rx=0 tx=0).stop 2023-08-25T04:45:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 msgr2=0x7fe7dc073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e9a0 tx=0x7fe7cc00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 shutdown_connections 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe7c809fd50 0x7fe7c80a2210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7dc1024e0 0x7fe7dc073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7dc100950 0x7fe7dc072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 --2- 172.21.15.145:0/2309479553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7dc101b40 0x7fe7dc073250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 >> 172.21.15.145:0/2309479553 conn(0x7fe7dc0fbea0 msgr2=0x7fe7dc104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.076+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 shutdown_connections 2023-08-25T04:45:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.077+0000 7fe7e43ff700 1 -- 172.21.15.145:0/2309479553 wait complete. 2023-08-25T04:45:06.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-25T04:45:06.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 -- 172.21.15.145:0/4273775711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c103d30 msgr2=0x7f295c104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 --2- 172.21.15.145:0/4273775711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c103d30 0x7f295c104190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2950009920 tx=0x7f2950009c30 comp rx=0 tx=0).stop 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 -- 172.21.15.145:0/4273775711 shutdown_connections 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 --2- 172.21.15.145:0/4273775711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 --2- 172.21.15.145:0/4273775711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c102b40 0x7f295c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 --2- 172.21.15.145:0/4273775711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c103d30 0x7f295c104190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 -- 172.21.15.145:0/4273775711 >> 172.21.15.145:0/4273775711 conn(0x7f295c0fe0d0 msgr2=0x7f295c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 -- 172.21.15.145:0/4273775711 shutdown_connections 2023-08-25T04:45:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.299+0000 7f2961bff700 1 -- 172.21.15.145:0/4273775711 wait complete. 2023-08-25T04:45:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.301+0000 7f2961bff700 1 Processor -- start 2023-08-25T04:45:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 -- start start 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c103d30 0x7f295c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c19a3a0 con 0x7f295c103d30 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c19a510 con 0x7f295c102b40 2023-08-25T04:45:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f2961bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c19a6d0 con 0x7f295c1046d0 2023-08-25T04:45:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f295b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f295b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:58192/0 (socket says 172.21.15.145:58192) 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f295b7fe700 1 -- 172.21.15.145:0/580674430 learned_addr learned my addr 172.21.15.145:0/580674430 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f295affd700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c103d30 0x7f295c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.302+0000 7f295bfff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 -- 172.21.15.145:0/580674430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 msgr2=0x7f295c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 -- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c103d30 msgr2=0x7f295c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c103d30 0x7f295c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 -- 172.21.15.145:0/580674430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0095f0 con 0x7f295c1046d0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295b7fe700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f295bfff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c199d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f295400efc0 tx=0x7f295400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954018070 con 0x7f295c1046d0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f295400f040 con 0x7f295c1046d0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295400cd40 con 0x7f295c1046d0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.303+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2950009520 con 0x7f295c1046d0 2023-08-25T04:45:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.304+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a0f90 con 0x7f295c1046d0 2023-08-25T04:45:06.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.305+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c10a1c0 con 0x7f295c1046d0 2023-08-25T04:45:06.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.306+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) v1 ==== 134483+0+0 (secure 0 0 0) 0x7f2954003a00 con 0x7f295c1046d0 2023-08-25T04:45:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.308+0000 7f2958ff9700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 0x7f29440a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.308+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f29540cae50 con 0x7f295c1046d0 2023-08-25T04:45:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.308+0000 7f295b7fe700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 0x7f29440a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.309+0000 7f295b7fe700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 0x7f29440a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f294c005950 tx=0x7f294c009310 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:06.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.312+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f2954094b20 con 0x7f295c1046d0 2023-08-25T04:45:06.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.547+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 21) v1 ==== 134792+0+0 (secure 0 0 0) 0x7f2954003cb0 con 0x7f295c1046d0 2023-08-25T04:45:06.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.578+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f295c060d60 con 0x7f294409fd50 2023-08-25T04:45:06.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.586+0000 7f2958ff9700 1 -- 172.21.15.145:0/580674430 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f295c060d60 con 0x7f294409fd50 2023-08-25T04:45:06.587 INFO:teuthology.orchestra.run.smithi145.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-25T00:00:00", "created": "2023-08-25T04:45:05", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-08-25T04:45:06.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.588+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 msgr2=0x7f29440a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.588+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 0x7f29440a2210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f294c005950 tx=0x7f294c009310 comp rx=0 tx=0).stop 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.588+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 msgr2=0x7f295c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.588+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c199d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f295400efc0 tx=0x7f295400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 shutdown_connections 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f295c1046d0 0x7f295c199d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f294409fd50 0x7f29440a2210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f295c102b40 0x7f295c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 --2- 172.21.15.145:0/580674430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f295c103d30 0x7f295c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 >> 172.21.15.145:0/580674430 conn(0x7f295c0fe0d0 msgr2=0x7f295c107300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:06.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 shutdown_connections 2023-08-25T04:45:06.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.589+0000 7f2961bff700 1 -- 172.21.15.145:0/580674430 wait complete. 2023-08-25T04:45:06.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-25T04:45:06.815 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 -- 172.21.15.145:0/959599077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 msgr2=0x7fe9fc101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.815 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 --2- 172.21.15.145:0/959599077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc101f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009990 tx=0x7fe9ec009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.815 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 -- 172.21.15.145:0/959599077 shutdown_connections 2023-08-25T04:45:06.815 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 --2- 172.21.15.145:0/959599077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 0x7fe9fc108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 --2- 172.21.15.145:0/959599077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 --2- 172.21.15.145:0/959599077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc101f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 -- 172.21.15.145:0/959599077 >> 172.21.15.145:0/959599077 conn(0x7fe9fc0fbe00 msgr2=0x7fe9fc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:06.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 -- 172.21.15.145:0/959599077 shutdown_connections 2023-08-25T04:45:06.816 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.814+0000 7fea033ff700 1 -- 172.21.15.145:0/959599077 wait complete. 2023-08-25T04:45:06.817 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.815+0000 7fea033ff700 1 Processor -- start 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.815+0000 7fea033ff700 1 -- start start 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 0x7fe9fc073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0739f0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.818 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc073b60 con 0x7fe9fc102460 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea033ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1a0620 con 0x7fe9fc101ac0 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58754/0 (socket says 172.21.15.145:58754) 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 -- 172.21.15.145:0/522763790 learned_addr learned my addr 172.21.15.145:0/522763790 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0099a700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0199c700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 0x7fe9fc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 -- 172.21.15.145:0/522763790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 msgr2=0x7fe9fc072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 -- 172.21.15.145:0/522763790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 msgr2=0x7fe9fc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 0x7fe9fc073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.816+0000 7fea0119b700 1 -- 172.21.15.145:0/522763790 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8009520 con 0x7fe9fc1008d0 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fea0119b700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800ea50 tx=0x7fe9f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800cbd0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec0095f0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f800cd30 con 0x7fe9fc1008d0 2023-08-25T04:45:06.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8018680 con 0x7fe9fc1008d0 2023-08-25T04:45:06.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.817+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc1a0cd0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.818+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc10c0e0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.820+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 21) v1 ==== 134792+0+0 (secure 0 0 0) 0x7fe9f80188a0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.822+0000 7fe9f27fc700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 0x7fe9e80a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:06.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.822+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe9f8014070 con 0x7fe9fc1008d0 2023-08-25T04:45:06.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.822+0000 7fea0099a700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 0x7fe9e80a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:06.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.823+0000 7fea0099a700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 0x7fe9e80a2580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006010 tx=0x7fe9ec00b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:06.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: [25/Aug/2023:04:45:05] ENGINE Bus STARTING 2023-08-25T04:45:06.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: [25/Aug/2023:04:45:05] ENGINE Serving on http://172.21.15.145:8765 2023-08-25T04:45:06.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: [25/Aug/2023:04:45:05] ENGINE Serving on https://172.21.15.145:7150 2023-08-25T04:45:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: [25/Aug/2023:04:45:05] ENGINE Bus STARTED 2023-08-25T04:45:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:06.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:06 smithi145 ceph-mon[97681]: scrub summary: idle 2023-08-25T04:45:06.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:06.830+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fe9f80955f0 con 0x7fe9fc1008d0 2023-08-25T04:45:06.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: [25/Aug/2023:04:45:05] ENGINE Bus STARTING 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: [25/Aug/2023:04:45:05] ENGINE Serving on http://172.21.15.145:8765 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: [25/Aug/2023:04:45:05] ENGINE Serving on https://172.21.15.145:7150 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: [25/Aug/2023:04:45:05] ENGINE Bus STARTED 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:06.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:06 smithi162 ceph-mon[130941]: scrub summary: idle 2023-08-25T04:45:06.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: [25/Aug/2023:04:45:05] ENGINE Bus STARTING 2023-08-25T04:45:06.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: [25/Aug/2023:04:45:05] ENGINE Serving on http://172.21.15.145:8765 2023-08-25T04:45:06.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: [25/Aug/2023:04:45:05] ENGINE Serving on https://172.21.15.145:7150 2023-08-25T04:45:06.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: [25/Aug/2023:04:45:05] ENGINE Bus STARTED 2023-08-25T04:45:06.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:06.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: pgmap v3: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:06.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:06 smithi183 ceph-mon[131414]: scrub summary: idle 2023-08-25T04:45:07.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.106+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe9fc1a0ef0 con 0x7fe9e80a00c0 2023-08-25T04:45:07.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.115+0000 7fe9f27fc700 1 -- 172.21.15.145:0/522763790 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe9fc1a0ef0 con 0x7fe9e80a00c0 2023-08-25T04:45:07.116 INFO:teuthology.orchestra.run.smithi145.stdout:/ 1M 6M3h 2023-08-25T04:45:07.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 msgr2=0x7fe9e80a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:07.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 0x7fe9e80a2580 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006010 tx=0x7fe9ec00b260 comp rx=0 tx=0).stop 2023-08-25T04:45:07.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 msgr2=0x7fe9fc0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:07.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800ea50 tx=0x7fe9f800ed60 comp rx=0 tx=0).stop 2023-08-25T04:45:07.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 shutdown_connections 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9e80a00c0 0x7fe9e80a2580 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9fc102460 0x7fe9fc073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc1008d0 0x7fe9fc0746d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 --2- 172.21.15.145:0/522763790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc101ac0 0x7fe9fc072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 >> 172.21.15.145:0/522763790 conn(0x7fe9fc0fbe00 msgr2=0x7fe9fc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 shutdown_connections 2023-08-25T04:45:07.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:07.117+0000 7fea033ff700 1 -- 172.21.15.145:0/522763790 wait complete. 2023-08-25T04:45:07.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-08-25T04:45:07.220 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-08-25T04:45:07.238 INFO:teuthology.run_tasks:Running task exec... 2023-08-25T04:45:07.247 INFO:teuthology.task.exec:Executing custom commands... 2023-08-25T04:45:07.247 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:45:07.248 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-08-25T04:45:07.346 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:07.377 INFO:teuthology.orchestra.run.smithi145.stdout:Filesystem Size Used Avail Use% Mounted on 2023-08-25T04:45:07.377 INFO:teuthology.orchestra.run.smithi145.stdout:0@b24d70d4-4300-11ee-9b3d-001a4aab830c.cephfs=/ 1.1T 332M 1.1T 1% /tmp/cephfs 2023-08-25T04:45:07.381 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-08-25T04:45:07.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-08-25T04:45:07.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-08-25T04:45:07.704 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.704 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: mgrmap e21: x(active, since 2s), standbys: y, z 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.705 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:07 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:07.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: mgrmap e21: x(active, since 2s), standbys: y, z 2023-08-25T04:45:07.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:07 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: pgmap v4: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:07.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: mgrmap e21: x(active, since 2s), standbys: y, z 2023-08-25T04:45:07.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='client.25570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:07.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:07.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:07 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.830 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:08 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:08.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.913 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:08 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:08.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:08.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:08 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:09.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.829 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:09 smithi145 ceph-mon[97681]: mgrmap e22: x(active, since 5s), standbys: y, z 2023-08-25T04:45:09.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:09 smithi162 ceph-mon[130941]: mgrmap e22: x(active, since 5s), standbys: y, z 2023-08-25T04:45:09.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:45:09.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:45:09.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:45:09.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:45:09.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: pgmap v5: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:09.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:09.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:09 smithi183 ceph-mon[131414]: mgrmap e22: x(active, since 5s), standbys: y, z 2023-08-25T04:45:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:45:11.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:45:11.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:10 smithi162 ceph-mon[130941]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:45:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:11.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:45:11.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:45:11.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:10 smithi183 ceph-mon[131414]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:45:11.204 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.205 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:11.205 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.205 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.206 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.206 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:45:11.206 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:45:11.206 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:45:11.206 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:11.207 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:11.207 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: Updating smithi145:/etc/ceph/ceph.conf 2023-08-25T04:45:11.207 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: Updating smithi162:/etc/ceph/ceph.conf 2023-08-25T04:45:11.207 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:10 smithi145 ceph-mon[97681]: Updating smithi183:/etc/ceph/ceph.conf 2023-08-25T04:45:11.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-08-25T04:45:11.315 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-08-25T04:45:11.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-08-25T04:45:11.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-08-25T04:45:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/: 2023-08-25T04:45:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 .. 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 data 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data: 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:11.531 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 .. 2023-08-25T04:45:11.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-08-25T04:45:11.660 INFO:teuthology.orchestra.run.smithi145.stdout:[Service] 2023-08-25T04:45:11.660 INFO:teuthology.orchestra.run.smithi145.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-08-25T04:45:11.661 INFO:teuthology.orchestra.run.smithi145.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-08-25T04:45:11.669 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-08-25T04:45:11.881 INFO:teuthology.orchestra.run.smithi145.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-08-25T04:45:12.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:11 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:11 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.conf 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: pgmap v6: 129 pgs: 129 active+clean; 970 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: Updating smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:12.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:12.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:11 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:13.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:13.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:13.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:13.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:13.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:13.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2023-08-25T04:45:13.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: Updating smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/config/ceph.client.admin.keyring 2023-08-25T04:45:13.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:13.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:45:13.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:45:13.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:45:13.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:45:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:13 smithi145 ceph-mon[97681]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T04:45:14.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:13 smithi162 ceph-mon[130941]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T04:45:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:13 smithi183 ceph-mon[131414]: pgmap v7: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-08-25T04:45:15.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:15 smithi162 ceph-mon[130941]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-08-25T04:45:15.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:15 smithi183 ceph-mon[131414]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-08-25T04:45:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:15 smithi145 ceph-mon[97681]: pgmap v8: 129 pgs: 129 active+clean; 1.2 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-08-25T04:45:17.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:17 smithi162 ceph-mon[130941]: pgmap v9: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T04:45:17.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:17 smithi183 ceph-mon[131414]: pgmap v9: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T04:45:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:17 smithi145 ceph-mon[97681]: pgmap v9: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.0 MiB/s wr, 66 op/s 2023-08-25T04:45:19.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:19 smithi162 ceph-mon[130941]: pgmap v10: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-08-25T04:45:19.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:19 smithi183 ceph-mon[131414]: pgmap v10: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-08-25T04:45:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:19 smithi145 ceph-mon[97681]: pgmap v10: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-08-25T04:45:21.602 INFO:teuthology.orchestra.run.smithi145.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-08-25T04:45:21.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-08-25T04:45:21.718 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/: 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 .. 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 data 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.719 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data: 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:total 30 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 . 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 .. 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 base 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 25 04:45 global 2023-08-25T04:45:21.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 log 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_commit_ts 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_dynshmem 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Aug 25 04:45 pg_hba.conf 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 25 04:45 pg_ident.conf 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 pg_logical 2023-08-25T04:45:21.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 pg_multixact 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 pg_notify 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_replslot 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_serial 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_snapshots 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_stat 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_stat_tmp 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 pg_subtrans 2023-08-25T04:45:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_tblspc 2023-08-25T04:45:21.723 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 pg_twophase 2023-08-25T04:45:21.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 PG_VERSION 2023-08-25T04:45:21.723 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 pg_wal 2023-08-25T04:45:21.723 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 pg_xact 2023-08-25T04:45:21.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 25 04:45 postgresql.auto.conf 2023-08-25T04:45:21.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Aug 25 04:45 postgresql.conf 2023-08-25T04:45:21.724 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.724 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/base: 2023-08-25T04:45:21.725 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 . 2023-08-25T04:45:21.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 1 2023-08-25T04:45:21.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 12346 2023-08-25T04:45:21.726 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 12347 2023-08-25T04:45:21.726 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.726 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/base/1: 2023-08-25T04:45:21.726 INFO:teuthology.orchestra.run.smithi145.stdout:total 7225 2023-08-25T04:45:21.726 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 . 2023-08-25T04:45:21.727 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 .. 2023-08-25T04:45:21.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 112 2023-08-25T04:45:21.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 113 2023-08-25T04:45:21.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 25 04:45 12182 2023-08-25T04:45:21.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12182_fsm 2023-08-25T04:45:21.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12182_vm 2023-08-25T04:45:21.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12184 2023-08-25T04:45:21.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12186 2023-08-25T04:45:21.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187 2023-08-25T04:45:21.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12187_fsm 2023-08-25T04:45:21.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187_vm 2023-08-25T04:45:21.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12189 2023-08-25T04:45:21.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12191 2023-08-25T04:45:21.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192 2023-08-25T04:45:21.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12192_fsm 2023-08-25T04:45:21.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192_vm 2023-08-25T04:45:21.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12194 2023-08-25T04:45:21.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12196 2023-08-25T04:45:21.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197 2023-08-25T04:45:21.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12197_fsm 2023-08-25T04:45:21.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197_vm 2023-08-25T04:45:21.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12199 2023-08-25T04:45:21.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12201 2023-08-25T04:45:21.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202 2023-08-25T04:45:21.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12202_fsm 2023-08-25T04:45:21.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202_vm 2023-08-25T04:45:21.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12204 2023-08-25T04:45:21.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12206 2023-08-25T04:45:21.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207 2023-08-25T04:45:21.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12207_fsm 2023-08-25T04:45:21.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207_vm 2023-08-25T04:45:21.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12209 2023-08-25T04:45:21.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12211 2023-08-25T04:45:21.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12212 2023-08-25T04:45:21.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12214 2023-08-25T04:45:21.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12216 2023-08-25T04:45:21.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 25 04:45 1247 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1247_fsm 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1247_vm 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 25 04:45 1249 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1249_fsm 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1249_vm 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 25 04:45 1255 2023-08-25T04:45:21.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1255_fsm 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1255_vm 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 25 04:45 1259 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1259_fsm 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1259_vm 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417_vm 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418 2023-08-25T04:45:21.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418_vm 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 174 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 175 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2187 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224_vm 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328_vm 2023-08-25T04:45:21.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336_vm 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2337 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2600 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2600_fsm 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2600_vm 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601 2023-08-25T04:45:21.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2601_fsm 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601_vm 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2602 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2602_fsm 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2602_vm 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2603 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2603_fsm 2023-08-25T04:45:21.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2603_vm 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604_vm 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2605 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2605_fsm 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2605_vm 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2606_fsm 2023-08-25T04:45:21.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606_vm 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607_fsm 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2607_vm 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 25 04:45 2608 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2608_fsm 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2608_vm 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 25 04:45 2609 2023-08-25T04:45:21.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2609_fsm 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2609_vm 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610_fsm 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2610_vm 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611_vm 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612 2023-08-25T04:45:21.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2612_fsm 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612_vm 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613_vm 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2615_fsm 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615_vm 2023-08-25T04:45:21.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616_fsm 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2616_vm 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 25 04:45 2617 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2617_fsm 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2617_vm 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 25 04:45 2618 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2618_fsm 2023-08-25T04:45:21.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2618_vm 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 25 04:45 2619 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2619_fsm 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2619_vm 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620_vm 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2650 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2651 2023-08-25T04:45:21.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2652 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2653 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2654 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2655 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2656 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2657 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 25 04:45 2658 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2659 2023-08-25T04:45:21.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2660 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2661 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2662 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2663 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2664 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2665 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2666 2023-08-25T04:45:21.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2667 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2668 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2669 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2670 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 25 04:45 2673 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 25 04:45 2674 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 25 04:45 2675 2023-08-25T04:45:21.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2678 2023-08-25T04:45:21.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2679 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2680 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2681 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2682 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2683 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2684 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2685 2023-08-25T04:45:21.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2686 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2687 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2688 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2689 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2690 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 25 04:45 2691 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2692 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2693 2023-08-25T04:45:21.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2696 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2699 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2701 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2702 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2703 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2704 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2753 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2753_fsm 2023-08-25T04:45:21.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2753_vm 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2754 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2755 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2756 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2757 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830_vm 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2831 2023-08-25T04:45:21.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832_vm 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2833 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834_vm 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2835 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836 2023-08-25T04:45:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836_vm 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2837 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 25 04:45 2838 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2838_fsm 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2838_vm 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2839 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840_fsm 2023-08-25T04:45:21.757 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2840_vm 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2841 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995_vm 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2996 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3079_fsm 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079_vm 2023-08-25T04:45:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3080 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3081 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3085 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118_vm 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3119 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3164 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256 2023-08-25T04:45:21.759 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256_vm 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3257 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3258 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350_vm 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3351 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3379 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3380 2023-08-25T04:45:21.760 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381_vm 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3394 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3394_fsm 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3394_vm 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3395 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439_vm 2023-08-25T04:45:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3440 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 3455 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3456 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3456_fsm 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3456_vm 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466 2023-08-25T04:45:21.762 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466_vm 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3467 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3468 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501_vm 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3502 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3503 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3534 2023-08-25T04:45:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3541_fsm 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541_vm 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3542 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3574 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3575 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576_vm 2023-08-25T04:45:21.764 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596_vm 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3597 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598_vm 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3599 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600 2023-08-25T04:45:21.765 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3600_fsm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600_vm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3601_fsm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601_vm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3602_fsm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602_vm 2023-08-25T04:45:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3603 2023-08-25T04:45:21.767 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3603_fsm 2023-08-25T04:45:21.767 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3603_vm 2023-08-25T04:45:21.767 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3604 2023-08-25T04:45:21.767 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3605 2023-08-25T04:45:21.767 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3606 2023-08-25T04:45:21.769 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3607 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3608 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 3609 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3712 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3764_fsm 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764_vm 2023-08-25T04:45:21.770 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3766 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3767 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3997 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 5002 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 548 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 549 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102 2023-08-25T04:45:21.771 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102_vm 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104_vm 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106_vm 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6110 2023-08-25T04:45:21.772 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6111 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6112 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6113 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6117 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826_vm 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 827 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 828 2023-08-25T04:45:21.773 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 25 04:45 pg_filenode.map 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 PG_VERSION 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:total 7225 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 . 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 .. 2023-08-25T04:45:21.774 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 112 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 113 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 25 04:45 12182 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12182_fsm 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12182_vm 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12184 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12186 2023-08-25T04:45:21.775 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12187_fsm 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187_vm 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12189 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12191 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12192_fsm 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192_vm 2023-08-25T04:45:21.776 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12194 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12196 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12197_fsm 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197_vm 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12199 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12201 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202 2023-08-25T04:45:21.777 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12202_fsm 2023-08-25T04:45:21.778 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202_vm 2023-08-25T04:45:21.778 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12204 2023-08-25T04:45:21.778 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12206 2023-08-25T04:45:21.778 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207 2023-08-25T04:45:21.778 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12207_fsm 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207_vm 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12209 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12211 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12212 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12214 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12216 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 25 04:45 1247 2023-08-25T04:45:21.779 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1247_fsm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1247_vm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 25 04:45 1249 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1249_fsm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1249_vm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 25 04:45 1255 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1255_fsm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1255_vm 2023-08-25T04:45:21.780 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 25 04:45 1259 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1259_fsm 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1259_vm 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417_vm 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418_vm 2023-08-25T04:45:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 174 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 175 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2187 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224_vm 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328_vm 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336 2023-08-25T04:45:21.782 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336_vm 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2337 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2600 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2600_fsm 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2600_vm 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601 2023-08-25T04:45:21.783 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2601_fsm 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601_vm 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2602 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2602_fsm 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2602_vm 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2603 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2603_fsm 2023-08-25T04:45:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2603_vm 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604_vm 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2605 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2605_fsm 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2605_vm 2023-08-25T04:45:21.785 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606 2023-08-25T04:45:21.786 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2606_fsm 2023-08-25T04:45:21.786 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606_vm 2023-08-25T04:45:21.786 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607 2023-08-25T04:45:21.786 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607_fsm 2023-08-25T04:45:21.786 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2607_vm 2023-08-25T04:45:21.787 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 25 04:45 2608 2023-08-25T04:45:21.787 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2608_fsm 2023-08-25T04:45:21.787 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2608_vm 2023-08-25T04:45:21.787 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 25 04:45 2609 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2609_fsm 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2609_vm 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610_fsm 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2610_vm 2023-08-25T04:45:21.788 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611_vm 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2612_fsm 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612_vm 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613_vm 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615 2023-08-25T04:45:21.789 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2615_fsm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615_vm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616_fsm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2616_vm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 25 04:45 2617 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2617_fsm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2617_vm 2023-08-25T04:45:21.790 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 25 04:45 2618 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2618_fsm 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2618_vm 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 25 04:45 2619 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2619_fsm 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2619_vm 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620_vm 2023-08-25T04:45:21.791 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2650 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2651 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2652 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2653 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2654 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2655 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2656 2023-08-25T04:45:21.792 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2657 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 25 04:45 2658 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2659 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2660 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2661 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2662 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2663 2023-08-25T04:45:21.793 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2664 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2665 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2666 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2667 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2668 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2669 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2670 2023-08-25T04:45:21.794 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 25 04:45 2673 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 25 04:45 2674 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 25 04:45 2675 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2678 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2679 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2680 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2681 2023-08-25T04:45:21.795 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2682 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2683 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2684 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2685 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2686 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2687 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2688 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2689 2023-08-25T04:45:21.796 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2690 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 25 04:45 2691 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2692 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2693 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2696 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2699 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2701 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2702 2023-08-25T04:45:21.797 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2703 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2704 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2753 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2753_fsm 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2753_vm 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2754 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2755 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2756 2023-08-25T04:45:21.798 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2757 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830_vm 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2831 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832_vm 2023-08-25T04:45:21.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2833 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834_vm 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2835 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836_vm 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2837 2023-08-25T04:45:21.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 25 04:45 2838 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2838_fsm 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2838_vm 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2839 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840_fsm 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2840_vm 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2841 2023-08-25T04:45:21.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995_vm 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2996 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3079_fsm 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079_vm 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3080 2023-08-25T04:45:21.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3081 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3085 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118_vm 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3119 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3164 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256 2023-08-25T04:45:21.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256_vm 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3257 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3258 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350_vm 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3351 2023-08-25T04:45:21.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3379 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3380 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381_vm 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3394 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3394_fsm 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3394_vm 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3395 2023-08-25T04:45:21.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439_vm 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3440 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 3455 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3456 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3456_fsm 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3456_vm 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466 2023-08-25T04:45:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466_vm 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3467 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3468 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501_vm 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3502 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3503 2023-08-25T04:45:21.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3534 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3541_fsm 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541_vm 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3542 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3574 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3575 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576 2023-08-25T04:45:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576_vm 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596_vm 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3597 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598_vm 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3599 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600 2023-08-25T04:45:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3600_fsm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600_vm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3601_fsm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601_vm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3602_fsm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602_vm 2023-08-25T04:45:21.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3603 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3603_fsm 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3603_vm 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3604 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3605 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3606 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3607 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3608 2023-08-25T04:45:21.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 3609 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3712 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3764_fsm 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764_vm 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3766 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3767 2023-08-25T04:45:21.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3997 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 5002 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 548 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 549 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102_vm 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104_vm 2023-08-25T04:45:21.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106_vm 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6110 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6111 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6112 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6113 2023-08-25T04:45:21.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6117 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826_vm 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 827 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 828 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 25 04:45 pg_filenode.map 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 PG_VERSION 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.815 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:total 7225 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 25 04:45 . 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 .. 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 112 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 113 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 25 04:45 12182 2023-08-25T04:45:21.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12182_fsm 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12182_vm 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12184 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12186 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12187_fsm 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12187_vm 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12189 2023-08-25T04:45:21.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12191 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12192_fsm 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12192_vm 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12194 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12196 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197 2023-08-25T04:45:21.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12197_fsm 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12197_vm 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12199 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12201 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12202_fsm 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12202_vm 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12204 2023-08-25T04:45:21.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12206 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 12207_fsm 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12207_vm 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12209 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12211 2023-08-25T04:45:21.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12212 2023-08-25T04:45:21.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 12214 2023-08-25T04:45:21.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 12216 2023-08-25T04:45:21.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 25 04:45 1247 2023-08-25T04:45:21.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1247_fsm 2023-08-25T04:45:21.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1247_vm 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 25 04:45 1249 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1249_fsm 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1249_vm 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 25 04:45 1255 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1255_fsm 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1255_vm 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 25 04:45 1259 2023-08-25T04:45:21.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1259_fsm 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1259_vm 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1417_vm 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 1418_vm 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 174 2023-08-25T04:45:21.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 175 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2187 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2224_vm 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2328_vm 2023-08-25T04:45:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2336_vm 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2337 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2600 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2600_fsm 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2600_vm 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601 2023-08-25T04:45:21.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2601_fsm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2601_vm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2602 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2602_fsm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2602_vm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2603 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2603_fsm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2603_vm 2023-08-25T04:45:21.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2604_vm 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2605 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2605_fsm 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2605_vm 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2606_fsm 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2606_vm 2023-08-25T04:45:21.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2607_fsm 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2607_vm 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 25 04:45 2608 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2608_fsm 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2608_vm 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 25 04:45 2609 2023-08-25T04:45:21.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2609_fsm 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2609_vm 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2610_fsm 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2610_vm 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2611_vm 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612 2023-08-25T04:45:21.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2612_fsm 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2612_vm 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2613_vm 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2615_fsm 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2615_vm 2023-08-25T04:45:21.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2616_fsm 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2616_vm 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 25 04:45 2617 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2617_fsm 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2617_vm 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 25 04:45 2618 2023-08-25T04:45:21.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2618_fsm 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2618_vm 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 25 04:45 2619 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2619_fsm 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2619_vm 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2620_vm 2023-08-25T04:45:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2650 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2651 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2652 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2653 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2654 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2655 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2656 2023-08-25T04:45:21.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2657 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 25 04:45 2658 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2659 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2660 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2661 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2662 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2663 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2664 2023-08-25T04:45:21.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2665 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2666 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2667 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2668 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2669 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2670 2023-08-25T04:45:21.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 25 04:45 2673 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 25 04:45 2674 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 25 04:45 2675 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2678 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2679 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2680 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2681 2023-08-25T04:45:21.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2682 2023-08-25T04:45:21.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2683 2023-08-25T04:45:21.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2684 2023-08-25T04:45:21.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2685 2023-08-25T04:45:21.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2686 2023-08-25T04:45:21.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2687 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2688 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 25 04:45 2689 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 25 04:45 2690 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 25 04:45 2691 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2692 2023-08-25T04:45:21.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2693 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2696 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2699 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2701 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2702 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2703 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 2704 2023-08-25T04:45:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2753 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2753_fsm 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2753_vm 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2754 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2755 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2756 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 2757 2023-08-25T04:45:21.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2830_vm 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2831 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2832_vm 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2833 2023-08-25T04:45:21.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2834_vm 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2835 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2836_vm 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2837 2023-08-25T04:45:21.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 25 04:45 2838 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2838_fsm 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2838_vm 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2839 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2840_fsm 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2840_vm 2023-08-25T04:45:21.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2841 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2995_vm 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2996 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3079_fsm 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3079_vm 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3080 2023-08-25T04:45:21.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3081 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3085 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3118_vm 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3119 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3164 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3256_vm 2023-08-25T04:45:21.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3257 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3258 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3350_vm 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3351 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3379 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3380 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381 2023-08-25T04:45:21.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3381_vm 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3394 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3394_fsm 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3394_vm 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3395 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3439_vm 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3440 2023-08-25T04:45:21.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 25 04:45 3455 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3456 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3456_fsm 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3456_vm 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3466_vm 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3467 2023-08-25T04:45:21.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3468 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3501_vm 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3502 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3503 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3534 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3541_fsm 2023-08-25T04:45:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3541_vm 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3542 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3574 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3575 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3576_vm 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3596_vm 2023-08-25T04:45:21.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3597 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3598_vm 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3599 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3600_fsm 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3600_vm 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601 2023-08-25T04:45:21.851 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3601_fsm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3601_vm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3602_fsm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3602_vm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3603 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3603_fsm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3603_vm 2023-08-25T04:45:21.852 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3604 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3605 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3606 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3607 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3608 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 25 04:45 3609 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3712 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764 2023-08-25T04:45:21.853 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 3764_fsm 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3764_vm 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3766 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 3767 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3997 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 5002 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 548 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 549 2023-08-25T04:45:21.854 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6102_vm 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6104_vm 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6106_vm 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6110 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6111 2023-08-25T04:45:21.855 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6112 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6113 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6117 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 826_vm 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 827 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 828 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 25 04:45 pg_filenode.map 2023-08-25T04:45:21.856 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 PG_VERSION 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/global: 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:total 553 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 25 04:45 . 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1136 2023-08-25T04:45:21.857 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1136_fsm 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1136_vm 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 1137 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1213 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1213_fsm 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1213_vm 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1214 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1214_fsm 2023-08-25T04:45:21.858 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1214_vm 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 1232 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 1233 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1260 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1260_fsm 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1260_vm 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1261 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1261_fsm 2023-08-25T04:45:21.859 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1261_vm 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1262 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 1262_fsm 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 1262_vm 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2396 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 25 04:45 2396_fsm 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2396_vm 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2397 2023-08-25T04:45:21.860 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2671 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2672 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2676 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2677 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2694 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2695 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2697 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 25 04:45 2698 2023-08-25T04:45:21.861 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2846 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2846_vm 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2847 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2964 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2964_vm 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2965 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2966 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 2966_vm 2023-08-25T04:45:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 2967 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3592 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 3592_vm 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 3593 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 4060 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 4060_vm 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 4061 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6000 2023-08-25T04:45:21.863 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6000_vm 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6001 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6002 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6100 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 6100_vm 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6114 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 6115 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 pg_control 2023-08-25T04:45:21.864 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 25 04:45 pg_filenode.map 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/log: 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-08-25T04:45:21.865 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.866 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:total 1 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 . 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 mappings 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 25 04:45 replorigin_checkpoint 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 snapshots 2023-08-25T04:45:21.867 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 .. 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-08-25T04:45:21.868 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 25 04:45 .. 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 . 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.869 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 members 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 offsets 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:total 8 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 .. 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 0000 2023-08-25T04:45:21.870 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:total 8 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 .. 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 0000 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-08-25T04:45:21.871 INFO:teuthology.orchestra.run.smithi145.stdout:total 8 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 0000 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-08-25T04:45:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.874 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:total 8 2023-08-25T04:45:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 0000 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-08-25T04:45:21.877 INFO:teuthology.orchestra.run.smithi145.stdout:total 16384 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 . 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 25 04:45 000000010000000000000001 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 archive_status 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:total 0 2023-08-25T04:45:21.878 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 25 04:45 . 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 25 04:45 .. 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:total 8 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 25 04:45 . 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 25 04:45 .. 2023-08-25T04:45:21.879 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 25 04:45 0000 2023-08-25T04:45:21.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-08-25T04:45:21.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:21 smithi162 ceph-mon[130941]: pgmap v11: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 61 op/s 2023-08-25T04:45:21.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:21 smithi183 ceph-mon[131414]: pgmap v11: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 61 op/s 2023-08-25T04:45:21.940 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:21 smithi145 ceph-mon[97681]: pgmap v11: 129 pgs: 129 active+clean; 22 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 61 op/s 2023-08-25T04:45:22.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-08-25T04:45:22.224 INFO:teuthology.orchestra.run.smithi145.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-08-25T04:45:22.230 INFO:teuthology.orchestra.run.smithi145.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-08-25T04:45:22.234 INFO:teuthology.orchestra.run.smithi145.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-08-25T04:45:22.240 INFO:teuthology.orchestra.run.smithi145.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-08-25T04:45:22.528 INFO:teuthology.orchestra.run.smithi145.stderr:creating tables... 2023-08-25T04:45:22.632 INFO:teuthology.orchestra.run.smithi145.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.10 s, remaining 50.63 s) 2023-08-25T04:45:22.919 INFO:teuthology.orchestra.run.smithi145.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.39 s, remaining 96.78 s) 2023-08-25T04:45:23.185 INFO:teuthology.orchestra.run.smithi145.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.65 s, remaining 108.45 s) 2023-08-25T04:45:23.334 INFO:teuthology.orchestra.run.smithi145.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.80 s, remaining 99.69 s) 2023-08-25T04:45:23.605 INFO:teuthology.orchestra.run.smithi145.stderr:500000 of 50000000 tuples (1%) done (elapsed 1.08 s, remaining 106.44 s) 2023-08-25T04:45:23.887 INFO:teuthology.orchestra.run.smithi145.stderr:600000 of 50000000 tuples (1%) done (elapsed 1.36 s, remaining 111.72 s) 2023-08-25T04:45:23.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:23 smithi162 ceph-mon[130941]: pgmap v12: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.9 MiB/s wr, 65 op/s 2023-08-25T04:45:23.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:23 smithi183 ceph-mon[131414]: pgmap v12: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.9 MiB/s wr, 65 op/s 2023-08-25T04:45:23.986 INFO:teuthology.orchestra.run.smithi145.stderr:700000 of 50000000 tuples (1%) done (elapsed 1.46 s, remaining 102.52 s) 2023-08-25T04:45:24.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:23 smithi145 ceph-mon[97681]: pgmap v12: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.9 MiB/s wr, 65 op/s 2023-08-25T04:45:24.311 INFO:teuthology.orchestra.run.smithi145.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.78 s, remaining 109.48 s) 2023-08-25T04:45:24.586 INFO:teuthology.orchestra.run.smithi145.stderr:900000 of 50000000 tuples (1%) done (elapsed 2.06 s, remaining 112.17 s) 2023-08-25T04:45:24.695 INFO:teuthology.orchestra.run.smithi145.stderr:1000000 of 50000000 tuples (2%) done (elapsed 2.17 s, remaining 106.10 s) 2023-08-25T04:45:24.978 INFO:teuthology.orchestra.run.smithi145.stderr:1100000 of 50000000 tuples (2%) done (elapsed 2.45 s, remaining 108.80 s) 2023-08-25T04:45:25.106 INFO:teuthology.orchestra.run.smithi145.stderr:1200000 of 50000000 tuples (2%) done (elapsed 2.58 s, remaining 104.74 s) 2023-08-25T04:45:25.376 INFO:teuthology.orchestra.run.smithi145.stderr:1300000 of 50000000 tuples (2%) done (elapsed 2.85 s, remaining 106.63 s) 2023-08-25T04:45:25.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:45:25.666 INFO:teuthology.orchestra.run.smithi145.stderr:1400000 of 50000000 tuples (2%) done (elapsed 3.14 s, remaining 108.87 s) 2023-08-25T04:45:25.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 -- 172.21.15.145:0/1673876285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe0101250 msgr2=0x7fbbe010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/1673876285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe0101250 0x7fbbe010d760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800b0a0 tx=0x7fbbd800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 -- 172.21.15.145:0/1673876285 shutdown_connections 2023-08-25T04:45:25.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/1673876285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe0101250 0x7fbbe010d760 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/1673876285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe01068a0 0x7fbbe0106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/1673876285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0100890 0x7fbbe0100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 -- 172.21.15.145:0/1673876285 >> 172.21.15.145:0/1673876285 conn(0x7fbbe00fc380 msgr2=0x7fbbe00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 -- 172.21.15.145:0/1673876285 shutdown_connections 2023-08-25T04:45:25.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.709+0000 7fbbe7bff700 1 -- 172.21.15.145:0/1673876285 wait complete. 2023-08-25T04:45:25.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.710+0000 7fbbe7bff700 1 Processor -- start 2023-08-25T04:45:25.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.710+0000 7fbbe7bff700 1 -- start start 2023-08-25T04:45:25.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0101250 0x7fbbe0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 0x7fbbe00736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01a9320 con 0x7fbbe01068a0 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01a9460 con 0x7fbbe0100890 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe7bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01a95a0 con 0x7fbbe0101250 2023-08-25T04:45:25.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45408/0 (socket says 172.21.15.145:45408) 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe599b700 1 -- 172.21.15.145:0/3906661614 learned_addr learned my addr 172.21.15.145:0/3906661614 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0101250 0x7fbbe0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe599b700 1 -- 172.21.15.145:0/3906661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0101250 msgr2=0x7fbbe0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.711+0000 7fbbe599b700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0101250 0x7fbbe0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe599b700 1 -- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 msgr2=0x7fbbe00736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe619c700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 0x7fbbe00736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe599b700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 0x7fbbe00736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe599b700 1 -- 172.21.15.145:0/3906661614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc009520 con 0x7fbbe0100890 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe619c700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 0x7fbbe00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe599b700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00eb30 tx=0x7fbbdc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbd6ffd700 1 -- 172.21.15.145:0/3906661614 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc009880 con 0x7fbbe0100890 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbd6ffd700 1 -- 172.21.15.145:0/3906661614 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbdc0044e0 con 0x7fbbe0100890 2023-08-25T04:45:25.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd8009d00 con 0x7fbbe0100890 2023-08-25T04:45:25.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbd6ffd700 1 -- 172.21.15.145:0/3906661614 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc003e30 con 0x7fbbe0100890 2023-08-25T04:45:25.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.712+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01a9c00 con 0x7fbbe0100890 2023-08-25T04:45:25.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.716+0000 7fbbd6ffd700 1 -- 172.21.15.145:0/3906661614 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7fbbdc004000 con 0x7fbbe0100890 2023-08-25T04:45:25.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.718+0000 7fbbd6ffd700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 0x7fbbcc0a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.718+0000 7fbbd6ffd700 1 -- 172.21.15.145:0/3906661614 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fbbdc014070 con 0x7fbbe0100890 2023-08-25T04:45:25.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.718+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 0x7fbbcc0a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.719+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 0x7fbbcc0a2690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00743c0 tx=0x7fbbd0009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.719+0000 7fbbfb48eb80 1 Processor -- start 2023-08-25T04:45:25.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.719+0000 7fbbfb48eb80 1 -- start start 2023-08-25T04:45:25.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71287b0 con 0x5580f71c6380 2023-08-25T04:45:25.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7119780 con 0x5580f71c6ca0 2023-08-25T04:45:25.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71170a0 con 0x5580f71c0c40 2023-08-25T04:45:25.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe519a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35492/0 (socket says 172.21.15.145:35492) 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe599b700 1 -- 172.21.15.145:0/3946819804 learned_addr learned my addr 172.21.15.145:0/3946819804 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe619c700 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe519a700 1 -- 172.21.15.145:0/3946819804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 msgr2=0x5580f71c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe519a700 1 -- 172.21.15.145:0/3946819804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 msgr2=0x5580f71c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.720+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbe519a700 1 -- 172.21.15.145:0/3946819804 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f6f92630 con 0x5580f71c6ca0 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbe599b700 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbe519a700 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c0700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0011c60 tx=0x7fbbd0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b4d30a5a44cd68 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbcbfff700 1 -- 172.21.15.145:0/3946819804 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0004230 con 0x5580f71c6ca0 2023-08-25T04:45:25.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbcbfff700 1 -- 172.21.15.145:0/3946819804 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd0021070 con 0x5580f71c6ca0 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.721+0000 7fbbcbfff700 1 -- 172.21.15.145:0/3946819804 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0003bc0 con 0x5580f71c6ca0 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/3946819804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 msgr2=0x5580f71c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c0700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0011c60 tx=0x7fbbd0014000 comp rx=0 tx=0).stop 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/3946819804 shutdown_connections 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/3946819804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c0700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/3946819804 >> 172.21.15.145:0/3946819804 conn(0x5580f71350e0 msgr2=0x5580f71bb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/3946819804 shutdown_connections 2023-08-25T04:45:25.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.722+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/3946819804 wait complete. 2023-08-25T04:45:25.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.723+0000 7fbbfb48eb80 1 -- start start 2023-08-25T04:45:25.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.723+0000 7fbbfb48eb80 1 Processor -- start 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7112990 con 0x5580f71c6ca0 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f70b0bb0 con 0x5580f71c6380 2023-08-25T04:45:25.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f70b08b0 con 0x5580f71c0c40 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbfb48eb80 10 client.0 _ll_register_callbacks cb 0x5580f6ffaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54080/0 (socket says 172.21.15.145:54080) 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 -- 172.21.15.145:0/1197163592 learned_addr learned my addr 172.21.15.145:0/1197163592 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe619c700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 -- 172.21.15.145:0/1197163592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 msgr2=0x5580f71c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 -- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 msgr2=0x5580f71ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe599b700 1 -- 172.21.15.145:0/1197163592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01a9c00 con 0x5580f71c0c40 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.724+0000 7fbbe619c700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc003fd0 tx=0x7fbbdc00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ae28f3ab4cb0279 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc099190 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580f71901c0 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fbbdc09cfc0 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbdc0a0fa0 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbfb48eb80 10 client.25591 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fbbd0017070 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580f71901c0 con 0x5580f71c0c40 2023-08-25T04:45:25.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.725+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbbdc0986f0 con 0x5580f71c0c40 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 10 client.25591 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 20 client.25591 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f71da680 0x5580f71dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 4 client.25591 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580f70b8ba0 con 0x5580f71da680 2023-08-25T04:45:25.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.726+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f71da680 0x5580f71dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.727+0000 7fbbc97fa700 10 client.25591 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.727+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f71da680 0x5580f71dcaa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471d6bd21caef03f server_cookie=15f16b2c61a11886 in_seq=0 out_seq=0 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.727+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1197163592 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580f70b8ba0 con 0x5580f71da680 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.727+0000 7fbbc97fa700 10 client.25591 handle_command_reply: tid=0 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 client.25591 shutdown 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 msgr2=0x5580f71ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc003fd0 tx=0x7fbbdc00cd10 comp rx=0 tx=0).stop 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 shutdown_connections 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f71da680 0x5580f71dcaa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71ccc50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1197163592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbc97fa700 0 client.25591 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 >> 172.21.15.145:0/1197163592 conn(0x5580f71350e0 msgr2=0x5580f71b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 shutdown_connections 2023-08-25T04:45:25.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.728+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1197163592 wait complete. 2023-08-25T04:45:25.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.729+0000 7fbbfb48eb80 20 client.25591 trim_cache size 0 max 16384 2023-08-25T04:45:25.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 Processor -- start 2023-08-25T04:45:25.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 -- start start 2023-08-25T04:45:25.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f6f94920 con 0x5580f71c6380 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71cb480 con 0x5580f71c6ca0 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.761+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71c8430 con 0x5580f71c0c40 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35516/0 (socket says 172.21.15.145:35516) 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 -- 172.21.15.145:0/781868834 learned_addr learned my addr 172.21.15.145:0/781868834 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 -- 172.21.15.145:0/781868834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 msgr2=0x5580f71cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 -- 172.21.15.145:0/781868834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 msgr2=0x5580f71c9bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c9bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.762+0000 7fbbe519a700 1 -- 172.21.15.145:0/781868834 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f71b4df0 con 0x5580f71c6380 2023-08-25T04:45:25.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.763+0000 7fbbe519a700 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e690 tx=0x7fbbd001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e674f3387a363b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.763+0000 7fbbd4bff700 1 -- 172.21.15.145:0/781868834 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000ec60 con 0x5580f71c6380 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.763+0000 7fbbd4bff700 1 -- 172.21.15.145:0/781868834 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd002b040 con 0x5580f71c6380 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.763+0000 7fbbd4bff700 1 -- 172.21.15.145:0/781868834 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0005540 con 0x5580f71c6380 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/781868834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 msgr2=0x5580f71c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e690 tx=0x7fbbd001f040 comp rx=0 tx=0).stop 2023-08-25T04:45:25.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/781868834 shutdown_connections 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6ca0 0x5580f71c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/781868834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6380 0x5580f71c9690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/781868834 >> 172.21.15.145:0/781868834 conn(0x5580f71350e0 msgr2=0x5580f71c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/781868834 shutdown_connections 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/781868834 wait complete. 2023-08-25T04:45:25.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 -- start start 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.764+0000 7fbbfb48eb80 1 Processor -- start 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71c2f60 con 0x5580f71c6ca0 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f71c30a0 con 0x5580f71c6380 2023-08-25T04:45:25.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f722e4b0 con 0x5580f71c0c40 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbfb48eb80 10 client.0 _ll_register_callbacks cb 0x7fbbdc098cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbe619c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbe519a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbe519a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45456/0 (socket says 172.21.15.145:45456) 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbe519a700 1 -- 172.21.15.145:0/1016231418 learned_addr learned my addr 172.21.15.145:0/1016231418 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.765+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 -- 172.21.15.145:0/1016231418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 msgr2=0x5580f71d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 -- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 msgr2=0x5580f71c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 -- 172.21.15.145:0/1016231418 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580f6f92630 con 0x5580f71c6380 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbe519a700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000ea90 tx=0x7fbbd0005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c2de0de1f76d26 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0032bf0 con 0x5580f71c6380 2023-08-25T04:45:25.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580f71b38b0 con 0x5580f71c6380 2023-08-25T04:45:25.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fbbd0032d50 con 0x5580f71c6380 2023-08-25T04:45:25.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.766+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd0032040 con 0x5580f71c6380 2023-08-25T04:45:25.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 10 client.25580 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:25.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5580f6f3ede0 con 0x5580f71c6380 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580f71b38b0 con 0x5580f71c6380 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbbd0003680 con 0x5580f71c6380 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 10 client.25580 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 20 client.25580 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f722f7f0 0x5580f7231be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 4 client.25580 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:45:25.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5580f710b970 con 0x5580f722f7f0 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.767+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f722f7f0 0x5580f7231be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbc97fa700 10 client.25580 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbe599b700 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f722f7f0 0x5580f7231be0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25bfc7e40cece46 server_cookie=1b65a24eed6fb597 in_seq=0 out_seq=0 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbc97fa700 1 -- 172.21.15.145:0/1016231418 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5580f710b970 con 0x5580f722f7f0 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbc97fa700 10 client.25580 handle_command_reply: tid=0 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 client.25580 shutdown 2023-08-25T04:45:25.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 msgr2=0x5580f71c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000ea90 tx=0x7fbbd0005cd0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 shutdown_connections 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5580f71c6ca0 0x5580f71c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5580f722f7f0 0x5580f7231be0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f71c0c40 0x5580f71d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 --2- 172.21.15.145:0/1016231418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5580f71c6380 0x5580f71c2250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbc97fa700 0 client.25580 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 >> 172.21.15.145:0/1016231418 conn(0x5580f71350e0 msgr2=0x5580f71b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 shutdown_connections 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.768+0000 7fbbfb48eb80 1 -- 172.21.15.145:0/1016231418 wait complete. 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbfb48eb80 20 client.25580 trim_cache size 0 max 16384 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:45:25.777 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "no active scrubs running", 2023-08-25T04:45:25.778 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": {} 2023-08-25T04:45:25.778 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:45:25.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 msgr2=0x7fbbcc0a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 0x7fbbcc0a2690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00743c0 tx=0x7fbbd0009260 comp rx=0 tx=0).stop 2023-08-25T04:45:25.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 msgr2=0x7fbbe0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:25.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00eb30 tx=0x7fbbdc00ee40 comp rx=0 tx=0).stop 2023-08-25T04:45:25.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 shutdown_connections 2023-08-25T04:45:25.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbe01068a0 0x7fbbe00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbbcc0a01d0 0x7fbbcc0a2690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbe0100890 0x7fbbe0072c20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 --2- 172.21.15.145:0/3906661614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0101250 0x7fbbe0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:25.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 >> 172.21.15.145:0/3906661614 conn(0x7fbbe00fc380 msgr2=0x7fbbe01043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:25.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 shutdown_connections 2023-08-25T04:45:25.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:25.769+0000 7fbbe7bff700 1 -- 172.21.15.145:0/3906661614 wait complete. 2023-08-25T04:45:25.788 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-08-25T04:45:25.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:45:25.827 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:25 smithi145 ceph-mon[97681]: pgmap v13: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-08-25T04:45:25.841 INFO:teuthology.orchestra.run.smithi145.stderr:1500000 of 50000000 tuples (3%) done (elapsed 3.31 s, remaining 107.06 s) 2023-08-25T04:45:25.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:25 smithi162 ceph-mon[130941]: pgmap v13: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-08-25T04:45:25.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:25 smithi183 ceph-mon[131414]: pgmap v13: 129 pgs: 129 active+clean; 22 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.8 MiB/s wr, 26 op/s 2023-08-25T04:45:26.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 -- 172.21.15.145:0/1409018026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 msgr2=0x7f06280730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 --2- 172.21.15.145:0/1409018026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06280730a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0624009990 tx=0x7f0624009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 -- 172.21.15.145:0/1409018026 shutdown_connections 2023-08-25T04:45:26.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 --2- 172.21.15.145:0/1409018026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0628111700 0x7f0628113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 --2- 172.21.15.145:0/1409018026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06280730a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 --2- 172.21.15.145:0/1409018026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06280735e0 0x7f06281111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 -- 172.21.15.145:0/1409018026 >> 172.21.15.145:0/1409018026 conn(0x7f06280fc1e0 msgr2=0x7f06280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 -- 172.21.15.145:0/1409018026 shutdown_connections 2023-08-25T04:45:26.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.041+0000 7f062e19d700 1 -- 172.21.15.145:0/1409018026 wait complete. 2023-08-25T04:45:26.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.042+0000 7f062e19d700 1 Processor -- start 2023-08-25T04:45:26.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.042+0000 7f062e19d700 1 -- start start 2023-08-25T04:45:26.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.042+0000 7f062e19d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.042+0000 7f062e19d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 0x7f06281a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062e19d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 0x7f062819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062e19d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062819d210 con 0x7f0628111700 2023-08-25T04:45:26.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062e19d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062819d380 con 0x7f0628072cc0 2023-08-25T04:45:26.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062e19d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062819d540 con 0x7f06280735e0 2023-08-25T04:45:26.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062c99a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 0x7f06281a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45476/0 (socket says 172.21.15.145:45476) 2023-08-25T04:45:26.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 -- 172.21.15.145:0/2009755211 learned_addr learned my addr 172.21.15.145:0/2009755211 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d99c700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 0x7f062819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 -- 172.21.15.145:0/2009755211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 msgr2=0x7f06281a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 0x7f06281a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 -- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 msgr2=0x7f062819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 0x7f062819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 -- 172.21.15.145:0/2009755211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620009c40 con 0x7f0628072cc0 2023-08-25T04:45:26.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062c99a700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 0x7f06281a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:26.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d99c700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 0x7f062819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:45:26.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.043+0000 7f062d19b700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0624005910 tx=0x7f0624004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.044+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062401d070 con 0x7f0628072cc0 2023-08-25T04:45:26.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.044+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0624022470 con 0x7f0628072cc0 2023-08-25T04:45:26.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.044+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06240095f0 con 0x7f0628072cc0 2023-08-25T04:45:26.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.044+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624017650 con 0x7f0628072cc0 2023-08-25T04:45:26.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.044+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06281a98f0 con 0x7f0628072cc0 2023-08-25T04:45:26.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.045+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628067630 con 0x7f0628072cc0 2023-08-25T04:45:26.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.048+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f06240225e0 con 0x7f0628072cc0 2023-08-25T04:45:26.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.049+0000 7f061e7fc700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 0x7f06140a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.050+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f06240cce60 con 0x7f0628072cc0 2023-08-25T04:45:26.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.050+0000 7f062c99a700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 0x7f06140a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.050+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f06240964f0 con 0x7f0628072cc0 2023-08-25T04:45:26.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.051+0000 7f062c99a700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 0x7f06140a2700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0628113920 tx=0x7f0618009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.131 INFO:teuthology.orchestra.run.smithi145.stderr:1600000 of 50000000 tuples (3%) done (elapsed 3.60 s, remaining 108.91 s) 2023-08-25T04:45:26.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.322+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f062819dff0 con 0x7f0628072cc0 2023-08-25T04:45:26.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.323+0000 7f061e7fc700 1 -- 172.21.15.145:0/2009755211 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f0624099b10 con 0x7f0628072cc0 2023-08-25T04:45:26.386 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:26.386 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:43:16.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:45:26.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 msgr2=0x7f06140a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 0x7f06140a2700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0628113920 tx=0x7f0618009260 comp rx=0 tx=0).stop 2023-08-25T04:45:26.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 msgr2=0x7f06281a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0624005910 tx=0x7f0624004710 comp rx=0 tx=0).stop 2023-08-25T04:45:26.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 shutdown_connections 2023-08-25T04:45:26.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0628111700 0x7f062819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f06140a0240 0x7f06140a2700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0628072cc0 0x7f06281a2ae0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 --2- 172.21.15.145:0/2009755211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06280735e0 0x7f06281a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 >> 172.21.15.145:0/2009755211 conn(0x7f06280fc1e0 msgr2=0x7f06281129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 shutdown_connections 2023-08-25T04:45:26.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.328+0000 7f062e19d700 1 -- 172.21.15.145:0/2009755211 wait complete. 2023-08-25T04:45:26.391 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 23 2023-08-25T04:45:26.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-08-25T04:45:26.411 INFO:teuthology.orchestra.run.smithi145.stderr:1700000 of 50000000 tuples (3%) done (elapsed 3.88 s, remaining 110.25 s) 2023-08-25T04:45:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:1800000 of 50000000 tuples (3%) done (elapsed 4.05 s, remaining 108.53 s) 2023-08-25T04:45:26.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 -- 172.21.15.145:0/3963221754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 msgr2=0x7fe388102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 --2- 172.21.15.145:0/3963221754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe388102f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe374009990 tx=0x7fe374009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 -- 172.21.15.145:0/3963221754 shutdown_connections 2023-08-25T04:45:26.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 --2- 172.21.15.145:0/3963221754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388103460 0x7fe38810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 --2- 172.21.15.145:0/3963221754 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388108ab0 0x7fe388108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 --2- 172.21.15.145:0/3963221754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe388102f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 -- 172.21.15.145:0/3963221754 >> 172.21.15.145:0/3963221754 conn(0x7fe3880fe590 msgr2=0x7fe3881009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 -- 172.21.15.145:0/3963221754 shutdown_connections 2023-08-25T04:45:26.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.636+0000 7fe38edff700 1 -- 172.21.15.145:0/3963221754 wait complete. 2023-08-25T04:45:26.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 Processor -- start 2023-08-25T04:45:26.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 -- start start 2023-08-25T04:45:26.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388103460 0x7fe38819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 0x7fe388198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388199070 con 0x7fe388103460 2023-08-25T04:45:26.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881991e0 con 0x7fe388102aa0 2023-08-25T04:45:26.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.637+0000 7fe38edff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881993a0 con 0x7fe388108ab0 2023-08-25T04:45:26.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45486/0 (socket says 172.21.15.145:45486) 2023-08-25T04:45:26.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38d39c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 0x7fe388198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2827574848 learned_addr learned my addr 172.21.15.145:0/2827574848 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe387fff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388103460 0x7fe38819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2827574848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 msgr2=0x7fe388198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 0x7fe388198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388103460 msgr2=0x7fe38819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388103460 0x7fe38819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2827574848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740095f0 con 0x7fe388102aa0 2023-08-25T04:45:26.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38d39c700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 0x7fe388198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:26.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e9a0 tx=0x7fe37c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe385ffb700 1 -- 172.21.15.145:0/2827574848 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009790 con 0x7fe388102aa0 2023-08-25T04:45:26.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe385ffb700 1 -- 172.21.15.145:0/2827574848 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c010430 con 0x7fe388102aa0 2023-08-25T04:45:26.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe388199680 con 0x7fe388102aa0 2023-08-25T04:45:26.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.638+0000 7fe385ffb700 1 -- 172.21.15.145:0/2827574848 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c018680 con 0x7fe388102aa0 2023-08-25T04:45:26.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.639+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a5980 con 0x7fe388102aa0 2023-08-25T04:45:26.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.641+0000 7fe385ffb700 1 -- 172.21.15.145:0/2827574848 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7fe37c0105a0 con 0x7fe388102aa0 2023-08-25T04:45:26.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.643+0000 7fe385ffb700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 0x7fe3780a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.643+0000 7fe385ffb700 1 -- 172.21.15.145:0/2827574848 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe37c014070 con 0x7fe388102aa0 2023-08-25T04:45:26.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.643+0000 7fe3a26b4b80 1 Processor -- start 2023-08-25T04:45:26.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.643+0000 7fe387fff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 0x7fe3780a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.643+0000 7fe3a26b4b80 1 -- start start 2023-08-25T04:45:26.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe387fff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 0x7fe3780a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe388108130 tx=0x7fe374005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e84c7b0 con 0x56309e8e4c40 2023-08-25T04:45:26.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e83d780 con 0x56309e8eaca0 2023-08-25T04:45:26.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e83b0a0 con 0x56309e8ea380 2023-08-25T04:45:26.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54138/0 (socket says 172.21.15.145:54138) 2023-08-25T04:45:26.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2768786794 learned_addr learned my addr 172.21.15.145:0/2768786794 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe38d39c700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.644+0000 7fe387fff700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2768786794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 msgr2=0x56309e8e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2768786794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 msgr2=0x56309e8e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 -- 172.21.15.145:0/2768786794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309e6b6630 con 0x56309e8ea380 2023-08-25T04:45:26.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38d39c700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:45:26.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe37c095380 tx=0x7fe37c0988b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578a4a3311f2ec22 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe372ffd700 1 -- 172.21.15.145:0/2768786794 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c09ccb0 con 0x56309e8ea380 2023-08-25T04:45:26.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe372ffd700 1 -- 172.21.15.145:0/2768786794 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c099e90 con 0x56309e8ea380 2023-08-25T04:45:26.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.645+0000 7fe372ffd700 1 -- 172.21.15.145:0/2768786794 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c098370 con 0x56309e8ea380 2023-08-25T04:45:26.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/2768786794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 msgr2=0x56309e8ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe37c095380 tx=0x7fe37c0988b0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/2768786794 shutdown_connections 2023-08-25T04:45:26.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8ea380 0x56309e8ea760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/2768786794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/2768786794 >> 172.21.15.145:0/2768786794 conn(0x56309e8590e0 msgr2=0x56309e8df410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/2768786794 shutdown_connections 2023-08-25T04:45:26.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.646+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/2768786794 wait complete. 2023-08-25T04:45:26.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.647+0000 7fe3a26b4b80 1 -- start start 2023-08-25T04:45:26.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.647+0000 7fe3a26b4b80 1 Processor -- start 2023-08-25T04:45:26.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 0x56309e8ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e836990 con 0x56309e8e4c40 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e7d4bb0 con 0x56309e8ea380 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e7d48b0 con 0x56309e8eaca0 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe3a26b4b80 10 client.0 _ll_register_callbacks cb 0x56309e71ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:26.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35582/0 (socket says 172.21.15.145:35582) 2023-08-25T04:45:26.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38cb9b700 1 -- 172.21.15.145:0/506040058 learned_addr learned my addr 172.21.15.145:0/506040058 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 0x56309e8ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe387fff700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 -- 172.21.15.145:0/506040058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 msgr2=0x56309e8ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 -- 172.21.15.145:0/506040058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 msgr2=0x56309e8edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8edf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.648+0000 7fe38d39c700 1 -- 172.21.15.145:0/506040058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a5980 con 0x56309e8eaca0 2023-08-25T04:45:26.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe38d39c700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 0x56309e8ee9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3800042c0 tx=0x7fe3800042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42688e56b31071a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe367fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:26.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380004bb0 con 0x56309e8eaca0 2023-08-25T04:45:26.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56309e8b41c0 con 0x56309e8eaca0 2023-08-25T04:45:26.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe380004d10 con 0x56309e8eaca0 2023-08-25T04:45:26.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe380021890 con 0x56309e8eaca0 2023-08-25T04:45:26.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe3a26b4b80 10 client.25612 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:26.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56309e662de0 con 0x56309e8eaca0 2023-08-25T04:45:26.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.649+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56309e8b41c0 con 0x56309e8eaca0 2023-08-25T04:45:26.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe380007500 con 0x56309e8eaca0 2023-08-25T04:45:26.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:26.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 10 client.25612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-08-25T04:45:26.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 20 client.25612 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:26.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e8fe690 0x56309e900ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 4 client.25612 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:26.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56309e7dcba0 con 0x56309e8fe690 2023-08-25T04:45:26.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.650+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e8fe690 0x56309e900ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.651+0000 7fe367fff700 10 client.25612 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:45:26.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.651+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e8fe690 0x56309e900ab0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a40bbeff6e6799 server_cookie=9fc10cedf7714bb in_seq=0 out_seq=0 2023-08-25T04:45:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.651+0000 7fe367fff700 1 -- 172.21.15.145:0/506040058 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56309e7dcba0 con 0x56309e8fe690 2023-08-25T04:45:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.651+0000 7fe367fff700 10 client.25612 handle_command_reply: tid=0 2023-08-25T04:45:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.651+0000 7fe3a26b4b80 1 client.25612 shutdown 2023-08-25T04:45:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 msgr2=0x56309e8ee9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 0x56309e8ee9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3800042c0 tx=0x7fe3800042f0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 shutdown_connections 2023-08-25T04:45:26.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e8fe690 0x56309e900ab0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8eaca0 0x56309e8ee9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8e4c40 0x56309e8edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/506040058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe367fff700 0 client.25612 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:45:26.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 >> 172.21.15.145:0/506040058 conn(0x56309e8590e0 msgr2=0x56309e8d9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 shutdown_connections 2023-08-25T04:45:26.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/506040058 wait complete. 2023-08-25T04:45:26.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.652+0000 7fe3a26b4b80 20 client.25612 trim_cache size 0 max 16384 2023-08-25T04:45:26.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.668+0000 7fe3a26b4b80 1 Processor -- start 2023-08-25T04:45:26.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 -- start start 2023-08-25T04:45:26.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8eaca0 0x56309e8e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e6b8920 con 0x56309e8eaca0 2023-08-25T04:45:26.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e8f5c10 con 0x56309e8ea380 2023-08-25T04:45:26.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e8e0100 con 0x56309e8e4c40 2023-08-25T04:45:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe387fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.669+0000 7fe38d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8eaca0 0x56309e8e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54150/0 (socket says 172.21.15.145:54150) 2023-08-25T04:45:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 -- 172.21.15.145:0/527881802 learned_addr learned my addr 172.21.15.145:0/527881802 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe387fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45522/0 (socket says 172.21.15.145:45522) 2023-08-25T04:45:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 -- 172.21.15.145:0/527881802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 msgr2=0x56309e8ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 -- 172.21.15.145:0/527881802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8eaca0 msgr2=0x56309e8e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8eaca0 0x56309e8e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 -- 172.21.15.145:0/527881802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309e64bc60 con 0x56309e8e4c40 2023-08-25T04:45:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0971d0 tx=0x7fe37c010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af39f29b4df3fe9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe3737fe700 1 -- 172.21.15.145:0/527881802 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0c8eb0 con 0x56309e8e4c40 2023-08-25T04:45:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe3737fe700 1 -- 172.21.15.145:0/527881802 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c099e90 con 0x56309e8e4c40 2023-08-25T04:45:26.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.670+0000 7fe3737fe700 1 -- 172.21.15.145:0/527881802 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0b8320 con 0x56309e8e4c40 2023-08-25T04:45:26.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/527881802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 msgr2=0x56309e8f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0971d0 tx=0x7fe37c010c80 comp rx=0 tx=0).stop 2023-08-25T04:45:26.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/527881802 shutdown_connections 2023-08-25T04:45:26.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8eaca0 0x56309e8e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8f56d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/527881802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8ea380 0x56309e8ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/527881802 >> 172.21.15.145:0/527881802 conn(0x56309e8590e0 msgr2=0x56309e8df1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/527881802 shutdown_connections 2023-08-25T04:45:26.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.671+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/527881802 wait complete. 2023-08-25T04:45:26.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.672+0000 7fe3a26b4b80 1 -- start start 2023-08-25T04:45:26.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.672+0000 7fe3a26b4b80 1 Processor -- start 2023-08-25T04:45:26.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.672+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8ea380 0x56309e8e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e8e6d60 con 0x56309e8ea380 2023-08-25T04:45:26.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e8e6ed0 con 0x56309e8eaca0 2023-08-25T04:45:26.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309e8e7090 con 0x56309e8e4c40 2023-08-25T04:45:26.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe3a26b4b80 10 client.0 _ll_register_callbacks cb 0x7fe380003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:26.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38d39c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54156/0 (socket says 172.21.15.145:54156) 2023-08-25T04:45:26.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 -- 172.21.15.145:0/3226873849 learned_addr learned my addr 172.21.15.145:0/3226873849 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe387fff700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8ea380 0x56309e8e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 -- 172.21.15.145:0/3226873849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 msgr2=0x56309e8e6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 -- 172.21.15.145:0/3226873849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8ea380 msgr2=0x56309e8e62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8ea380 0x56309e8e62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 -- 172.21.15.145:0/3226873849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56309e6b6630 con 0x56309e8e4c40 2023-08-25T04:45:26.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38cb9b700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe37c098e10 tx=0x7fe37c095240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a4c31d797c513c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe38d39c700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:26.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.673+0000 7fe367fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:26.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c097220 con 0x56309e8e4c40 2023-08-25T04:45:26.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe37c0c8eb0 con 0x56309e8e4c40 2023-08-25T04:45:26.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56309e8d78b0 con 0x56309e8e4c40 2023-08-25T04:45:26.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37c0b80c0 con 0x56309e8e4c40 2023-08-25T04:45:26.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 10 client.25618 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:26.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56309e689250 con 0x56309e8e4c40 2023-08-25T04:45:26.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56309e8d78b0 con 0x56309e8e4c40 2023-08-25T04:45:26.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe37c07e080 con 0x56309e8e4c40 2023-08-25T04:45:26.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 10 client.25618 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:26.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 10 client.25618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-08-25T04:45:26.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 20 client.25618 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:26.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e9529c0 0x56309e954db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 4 client.25618 mds_command: new command op to 24532 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:45:26.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.674+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56309e82f970 con 0x56309e9529c0 2023-08-25T04:45:26.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe387fff700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e9529c0 0x56309e954db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe387fff700 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e9529c0 0x56309e954db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69beddd23cb0496a server_cookie=2dd58a5985867caf in_seq=0 out_seq=0 2023-08-25T04:45:26.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe367fff700 10 client.25618 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:45:26.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe367fff700 1 -- 172.21.15.145:0/3226873849 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56309e82f970 con 0x56309e9529c0 2023-08-25T04:45:26.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe367fff700 10 client.25618 handle_command_reply: tid=0 2023-08-25T04:45:26.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 client.25618 shutdown 2023-08-25T04:45:26.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 msgr2=0x56309e8e5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe37c098e10 tx=0x7fe37c095240 comp rx=0 tx=0).stop 2023-08-25T04:45:26.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 shutdown_connections 2023-08-25T04:45:26.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x56309e9529c0 0x56309e954db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56309e8eaca0 0x56309e8e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56309e8e4c40 0x56309e8e5da0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe367fff700 0 client.25618 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:45:26.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 --2- 172.21.15.145:0/3226873849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56309e8ea380 0x56309e8e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.675+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 >> 172.21.15.145:0/3226873849 conn(0x56309e8590e0 msgr2=0x56309e8e8050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 shutdown_connections 2023-08-25T04:45:26.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe3a26b4b80 1 -- 172.21.15.145:0/3226873849 wait complete. 2023-08-25T04:45:26.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe3a26b4b80 20 client.25618 trim_cache size 0 max 16384 2023-08-25T04:45:26.699 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:45:26.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 msgr2=0x7fe3780a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 0x7fe3780a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe388108130 tx=0x7fe374005fd0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 msgr2=0x7fe38819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e9a0 tx=0x7fe37c00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 shutdown_connections 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe388108ab0 0x7fe388198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe3780a0150 0x7fe3780a2610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe388102aa0 0x7fe38819e990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 --2- 172.21.15.145:0/2827574848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe388103460 0x7fe38819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.676+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 >> 172.21.15.145:0/2827574848 conn(0x7fe3880fe590 msgr2=0x7fe3880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.677+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 shutdown_connections 2023-08-25T04:45:26.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.677+0000 7fe38edff700 1 -- 172.21.15.145:0/2827574848 wait complete. 2023-08-25T04:45:26.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-08-25T04:45:26.843 INFO:teuthology.orchestra.run.smithi145.stderr:1900000 of 50000000 tuples (3%) done (elapsed 4.31 s, remaining 109.20 s) 2023-08-25T04:45:26.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:26 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2009755211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:45:26.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:26 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2009755211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:45:26.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.932+0000 7f989ffff700 1 -- 172.21.15.145:0/1102278947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898101a30 msgr2=0x7f9898101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.932+0000 7f989ffff700 1 --2- 172.21.15.145:0/1102278947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898101a30 0x7f9898101e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98940098f0 tx=0x7f9894009c00 comp rx=0 tx=0).stop 2023-08-25T04:45:26.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 -- 172.21.15.145:0/1102278947 shutdown_connections 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 --2- 172.21.15.145:0/1102278947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f989810ae50 0x7f989810d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 --2- 172.21.15.145:0/1102278947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898101a30 0x7f9898101e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 --2- 172.21.15.145:0/1102278947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9898102350 0x7f989810a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 -- 172.21.15.145:0/1102278947 >> 172.21.15.145:0/1102278947 conn(0x7f98980fb270 msgr2=0x7f98980fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 -- 172.21.15.145:0/1102278947 shutdown_connections 2023-08-25T04:45:26.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.933+0000 7f989ffff700 1 -- 172.21.15.145:0/1102278947 wait complete. 2023-08-25T04:45:26.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 Processor -- start 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 -- start start 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9898101a30 0x7f989819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898102350 0x7f989819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898196f70 con 0x7f9898101a30 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981970e0 con 0x7f9898102350 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989ffff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981972a0 con 0x7f989810ae50 2023-08-25T04:45:26.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.934+0000 7f989e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54166/0 (socket says 172.21.15.145:54166) 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9898101a30 0x7f989819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 -- 172.21.15.145:0/2927245746 learned_addr learned my addr 172.21.15.145:0/2927245746 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989d59a700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898102350 0x7f989819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 -- 172.21.15.145:0/2927245746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898102350 msgr2=0x7f989819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898102350 0x7f989819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 -- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9898101a30 msgr2=0x7f989819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9898101a30 0x7f989819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 -- 172.21.15.145:0/2927245746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009520 con 0x7f989810ae50 2023-08-25T04:45:26.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989e59c700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9890007ee0 tx=0x7f989000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f988effd700 1 -- 172.21.15.145:0/2927245746 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000dd20 con 0x7f989810ae50 2023-08-25T04:45:26.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f988effd700 1 -- 172.21.15.145:0/2927245746 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989000f040 con 0x7f989810ae50 2023-08-25T04:45:26.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9898197580 con 0x7f989810ae50 2023-08-25T04:45:26.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.935+0000 7f988effd700 1 -- 172.21.15.145:0/2927245746 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900048a0 con 0x7f989810ae50 2023-08-25T04:45:26.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.936+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98981a3820 con 0x7f989810ae50 2023-08-25T04:45:26.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.940+0000 7f988effd700 1 -- 172.21.15.145:0/2927245746 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f9890003a00 con 0x7f989810ae50 2023-08-25T04:45:26.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.941+0000 7f98b39a6b80 1 Processor -- start 2023-08-25T04:45:26.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.941+0000 7f98b39a6b80 1 -- start start 2023-08-25T04:45:26.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2eed00 0x55b84a2ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 0x55b84a2f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2527b0 con 0x55b84a2ef620 2023-08-25T04:45:26.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a243780 con 0x55b84a2f38d0 2023-08-25T04:45:26.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2410a0 con 0x55b84a2eed00 2023-08-25T04:45:26.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45550/0 (socket says 172.21.15.145:45550) 2023-08-25T04:45:26.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 -- 172.21.15.145:0/887060042 learned_addr learned my addr 172.21.15.145:0/887060042 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2eed00 0x55b84a2ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989dd9b700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 0x55b84a2f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 -- 172.21.15.145:0/887060042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2eed00 msgr2=0x55b84a2ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.942+0000 7f989d59a700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2eed00 0x55b84a2ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989d59a700 1 -- 172.21.15.145:0/887060042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 msgr2=0x55b84a2f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989d59a700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 0x55b84a2f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989d59a700 1 -- 172.21.15.145:0/887060042 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b84a0bc630 con 0x55b84a2f38d0 2023-08-25T04:45:26.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f988effd700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 0x7f98840a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989dd9b700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 0x55b84a2f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:45:26.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989d59a700 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9888011860 tx=0x7f98880058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3af9cc20e140d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f988effd700 1 -- 172.21.15.145:0/2927245746 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f98900cb2d0 con 0x7f989810ae50 2023-08-25T04:45:26.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f9883fff700 1 -- 172.21.15.145:0/887060042 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888004830 con 0x55b84a2f38d0 2023-08-25T04:45:26.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f9883fff700 1 -- 172.21.15.145:0/887060042 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9888004e50 con 0x55b84a2f38d0 2023-08-25T04:45:26.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f9883fff700 1 -- 172.21.15.145:0/887060042 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888005720 con 0x55b84a2f38d0 2023-08-25T04:45:26.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.943+0000 7f989dd9b700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 0x7f98840a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 -- 172.21.15.145:0/887060042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 msgr2=0x55b84a2f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9888011860 tx=0x7f98880058e0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f989dd9b700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 0x7f98840a2610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9894005e90 tx=0x7f9894005dc0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 -- 172.21.15.145:0/887060042 shutdown_connections 2023-08-25T04:45:26.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2f3d30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2eed00 0x55b84a2ef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/887060042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2ef620 0x55b84a2f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 -- 172.21.15.145:0/887060042 >> 172.21.15.145:0/887060042 conn(0x55b84a25f0e0 msgr2=0x55b84a2e5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 -- 172.21.15.145:0/887060042 shutdown_connections 2023-08-25T04:45:26.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.944+0000 7f98b39a6b80 1 -- 172.21.15.145:0/887060042 wait complete. 2023-08-25T04:45:26.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.945+0000 7f98b39a6b80 1 -- start start 2023-08-25T04:45:26.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.945+0000 7f98b39a6b80 1 Processor -- start 2023-08-25T04:45:26.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a23c990 con 0x55b84a2eed00 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a1dabb0 con 0x55b84a2ef620 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a1da8b0 con 0x55b84a2f38d0 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f98b39a6b80 10 client.0 _ll_register_callbacks cb 0x55b84a124a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35628/0 (socket says 172.21.15.145:35628) 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.946+0000 7f989dd9b700 1 -- 172.21.15.145:0/3237696403 learned_addr learned my addr 172.21.15.145:0/3237696403 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989e59c700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 -- 172.21.15.145:0/3237696403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 msgr2=0x55b84a2f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 -- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 msgr2=0x55b84a2f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989d59a700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 -- 172.21.15.145:0/3237696403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888019040 con 0x55b84a2eed00 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989d59a700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f989dd9b700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f98940098c0 tx=0x7f9894009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b163c69355b679 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.947+0000 7f98817fa700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894028070 con 0x55b84a2eed00 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9894005250 con 0x55b84a2eed00 2023-08-25T04:45:26.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b84a2ba1c0 con 0x55b84a2eed00 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9894005410 con 0x55b84a2eed00 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98b39a6b80 10 client.15612 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b84a068de0 con 0x55b84a2eed00 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.948+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b84a2ba1c0 con 0x55b84a2eed00 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9894021510 con 0x55b84a2eed00 2023-08-25T04:45:26.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 20 client.15612 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a308c80 0x55b84a30b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 4 client.15612 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b84a1e2ba0 con 0x55b84a308c80 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f989d59a700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a308c80 0x55b84a30b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f989d59a700 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a308c80 0x55b84a30b0a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efe4019504bb7c46 server_cookie=9ad4d8732c4c1640 in_seq=0 out_seq=0 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.949+0000 7f98817fa700 10 client.15612 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98817fa700 1 -- 172.21.15.145:0/3237696403 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b84a1e2ba0 con 0x55b84a308c80 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98817fa700 10 client.15612 handle_command_reply: tid=0 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 client.15612 shutdown 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 msgr2=0x55b84a2f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f98940098c0 tx=0x7f9894009c00 comp rx=0 tx=0).stop 2023-08-25T04:45:26.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 shutdown_connections 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a308c80 0x55b84a30b0a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f83c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/3237696403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.950+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 >> 172.21.15.145:0/3237696403 conn(0x55b84a25f0e0 msgr2=0x55b84a2df2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.951+0000 7f98817fa700 0 client.15612 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:45:26.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.951+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 shutdown_connections 2023-08-25T04:45:26.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.951+0000 7f98b39a6b80 1 -- 172.21.15.145:0/3237696403 wait complete. 2023-08-25T04:45:26.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.951+0000 7f98b39a6b80 20 client.15612 trim_cache size 0 max 16384 2023-08-25T04:45:26.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 Processor -- start 2023-08-25T04:45:26.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 -- start start 2023-08-25T04:45:26.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a0be920 con 0x55b84a2eed00 2023-08-25T04:45:26.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2fb9b0 con 0x55b84a2ef620 2023-08-25T04:45:26.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.969+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2f2510 con 0x55b84a2f38d0 2023-08-25T04:45:26.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35640/0 (socket says 172.21.15.145:35640) 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989dd9b700 1 -- 172.21.15.145:0/1580749519 learned_addr learned my addr 172.21.15.145:0/1580749519 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45570/0 (socket says 172.21.15.145:45570) 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989e59c700 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 -- 172.21.15.145:0/1580749519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 msgr2=0x55b84a2df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 -- 172.21.15.145:0/1580749519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 msgr2=0x55b84a2f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 -- 172.21.15.145:0/1580749519 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b84a2dedf0 con 0x55b84a2ef620 2023-08-25T04:45:26.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989dd9b700 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:26.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.970+0000 7f989d59a700 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9888001eb0 tx=0x7f98880097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a48abdc8932514 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.971+0000 7f988cff9700 1 -- 172.21.15.145:0/1580749519 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888024040 con 0x55b84a2ef620 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.971+0000 7f988cff9700 1 -- 172.21.15.145:0/1580749519 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9888005bc0 con 0x55b84a2ef620 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.971+0000 7f988cff9700 1 -- 172.21.15.145:0/1580749519 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988801e3f0 con 0x55b84a2ef620 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.971+0000 7f98b39a6b80 1 -- 172.21.15.145:0/1580749519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 msgr2=0x55b84a2f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.971+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9888001eb0 tx=0x7f98880097c0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 -- 172.21.15.145:0/1580749519 shutdown_connections 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2f38d0 0x55b84a2df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/1580749519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2ef620 0x55b84a2f8c70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 -- 172.21.15.145:0/1580749519 >> 172.21.15.145:0/1580749519 conn(0x55b84a25f0e0 msgr2=0x55b84a2f2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 -- 172.21.15.145:0/1580749519 shutdown_connections 2023-08-25T04:45:26.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 -- 172.21.15.145:0/1580749519 wait complete. 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 -- start start 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.972+0000 7f98b39a6b80 1 Processor -- start 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2ef620 0x55b84a2ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2fe090 con 0x55b84a2eed00 2023-08-25T04:45:26.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2fe200 con 0x55b84a2f38d0 2023-08-25T04:45:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b84a2fe3c0 con 0x55b84a2ef620 2023-08-25T04:45:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f98b39a6b80 10 client.0 _ll_register_callbacks cb 0x7f989400b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35648/0 (socket says 172.21.15.145:35648) 2023-08-25T04:45:26.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 -- 172.21.15.145:0/2367446447 learned_addr learned my addr 172.21.15.145:0/2367446447 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989d59a700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2ef620 0x55b84a2ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989e59c700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 -- 172.21.15.145:0/2367446447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2ef620 msgr2=0x55b84a2ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2ef620 0x55b84a2ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 -- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 msgr2=0x55b84a2fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:26.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.973+0000 7f989dd9b700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f989dd9b700 1 -- 172.21.15.145:0/2367446447 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b84a068de0 con 0x55b84a2eed00 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f989dd9b700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894021410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826304615bdf3853 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f98817fa700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894028040 con 0x55b84a2eed00 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9894005660 con 0x55b84a2eed00 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b84a2dd8b0 con 0x55b84a2eed00 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.974+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98940216b0 con 0x55b84a2eed00 2023-08-25T04:45:26.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 10 client.15618 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b84a0bc630 con 0x55b84a2eed00 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9894045080 con 0x55b84a2eed00 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9894034070 con 0x55b84a2eed00 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 20 client.15618 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a35d800 0x55b84a35fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:26.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 4 client.15618 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:45:26.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.975+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b84a235970 con 0x55b84a35d800 2023-08-25T04:45:26.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.976+0000 7f989d59a700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a35d800 0x55b84a35fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:26.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f989d59a700 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a35d800 0x55b84a35fbf0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af85250f5e766636 server_cookie=e41733db4e41ca67 in_seq=0 out_seq=0 2023-08-25T04:45:26.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98817fa700 10 client.15618 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:45:26.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98817fa700 1 -- 172.21.15.145:0/2367446447 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b84a235970 con 0x55b84a35d800 2023-08-25T04:45:26.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98817fa700 10 client.15618 handle_command_reply: tid=0 2023-08-25T04:45:26.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98b39a6b80 1 client.15618 shutdown 2023-08-25T04:45:26.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 msgr2=0x55b84a2ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.978+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894021410 comp rx=0 tx=0).stop 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 shutdown_connections 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x55b84a35d800 0x55b84a35fbf0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b84a2f38d0 0x55b84a2fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b84a2eed00 0x55b84a2ff630 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98817fa700 0 client.15618 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:45:26.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 --2- 172.21.15.145:0/2367446447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b84a2ef620 0x55b84a2ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 >> 172.21.15.145:0/2367446447 conn(0x55b84a25f0e0 msgr2=0x55b84a2f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 shutdown_connections 2023-08-25T04:45:26.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 1 -- 172.21.15.145:0/2367446447 wait complete. 2023-08-25T04:45:26.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.979+0000 7f98b39a6b80 20 client.15618 trim_cache size 0 max 16384 2023-08-25T04:45:26.989 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:45:26.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 msgr2=0x7f98840a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 0x7f98840a2610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9894005e90 tx=0x7f9894005dc0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 msgr2=0x7f9898196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9890007ee0 tx=0x7f989000d3b0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 shutdown_connections 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f989810ae50 0x7f9898196910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9898101a30 0x7f989819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f98840a0150 0x7f98840a2610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 --2- 172.21.15.145:0/2927245746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9898102350 0x7f989819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:26.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.980+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 >> 172.21.15.145:0/2927245746 conn(0x7f98980fb270 msgr2=0x7f989810d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:26.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.981+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 shutdown_connections 2023-08-25T04:45:26.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:26.981+0000 7f989ffff700 1 -- 172.21.15.145:0/2927245746 wait complete. 2023-08-25T04:45:27.006 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2009755211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:45:27.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-08-25T04:45:27.129 INFO:teuthology.orchestra.run.smithi145.stderr:2000000 of 50000000 tuples (4%) done (elapsed 4.60 s, remaining 110.36 s) 2023-08-25T04:45:27.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.234+0000 7ff8f999d700 1 -- 172.21.15.145:0/2578882376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4067a80 msgr2=0x7ff8f4067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.235 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.234+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2578882376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4067a80 0x7ff8f4067f00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009990 tx=0x7ff8e0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 -- 172.21.15.145:0/2578882376 shutdown_connections 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2578882376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4068440 0x7ff8f410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2578882376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4106780 0x7ff8f4106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2578882376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4067a80 0x7ff8f4067f00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 -- 172.21.15.145:0/2578882376 >> 172.21.15.145:0/2578882376 conn(0x7ff8f40750e0 msgr2=0x7ff8f40754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.236 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 -- 172.21.15.145:0/2578882376 shutdown_connections 2023-08-25T04:45:27.237 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.235+0000 7ff8f999d700 1 -- 172.21.15.145:0/2578882376 wait complete. 2023-08-25T04:45:27.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 Processor -- start 2023-08-25T04:45:27.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 -- start start 2023-08-25T04:45:27.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4067a80 0x7ff8f419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 0x7ff8f4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4198df0 con 0x7ff8f4068440 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4198f60 con 0x7ff8f4067a80 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.236+0000 7ff8f999d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4199120 con 0x7ff8f4106780 2023-08-25T04:45:27.239 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35672/0 (socket says 172.21.15.145:35672) 2023-08-25T04:45:27.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2762722672 learned_addr learned my addr 172.21.15.145:0/2762722672 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:27.240 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f899b700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4067a80 0x7ff8f419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.240 INFO:teuthology.orchestra.run.smithi145.stderr:2100000 of 50000000 tuples (4%) done (elapsed 4.71 s, remaining 107.37 s) 2023-08-25T04:45:27.241 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f919c700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 0x7ff8f4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2762722672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 msgr2=0x7ff8f4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 0x7ff8f4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2762722672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4067a80 msgr2=0x7ff8f419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4067a80 0x7ff8f419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.242 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2762722672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00095f0 con 0x7ff8f4068440 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f919c700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 0x7ff8f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.237+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.238+0000 7ff8f1ffb700 1 -- 172.21.15.145:0/2762722672 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e001d070 con 0x7ff8f4068440 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.238+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f41993a0 con 0x7ff8f4068440 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.238+0000 7ff8f1ffb700 1 -- 172.21.15.145:0/2762722672 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e0022470 con 0x7ff8f4068440 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.238+0000 7ff8f1ffb700 1 -- 172.21.15.145:0/2762722672 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0017650 con 0x7ff8f4068440 2023-08-25T04:45:27.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.238+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f41a5700 con 0x7ff8f4068440 2023-08-25T04:45:27.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.241+0000 7ff8f1ffb700 1 -- 172.21.15.145:0/2762722672 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7ff8e0022850 con 0x7ff8f4068440 2023-08-25T04:45:27.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.242+0000 7ff90e4d1b80 1 Processor -- start 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.242+0000 7ff90e4d1b80 1 -- start start 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b8068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b801080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b7687b0 con 0x55688b800c40 2023-08-25T04:45:27.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b759780 con 0x55688b806500 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b7570a0 con 0x55688b806e20 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b801080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f919c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b8068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54230/0 (socket says 172.21.15.145:54230) 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 -- 172.21.15.145:0/3774976290 learned_addr learned my addr 172.21.15.145:0/3774976290 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:27.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 -- 172.21.15.145:0/3774976290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 msgr2=0x55688b8068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b8068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 -- 172.21.15.145:0/3774976290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 msgr2=0x55688b801080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b801080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f899b700 1 -- 172.21.15.145:0/3774976290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55688b5d2630 con 0x55688b806e20 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.243+0000 7ff8f919c700 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b8068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8f1ffb700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 0x7ff8e40a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009d00 tx=0x7ff8e8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d657a000cc382fa4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8deffd700 1 -- 172.21.15.145:0/3774976290 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004230 con 0x55688b806e20 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8f1ffb700 1 -- 172.21.15.145:0/2762722672 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7ff8e00ccf00 con 0x7ff8f4068440 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8f919c700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 0x7ff8e40a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8deffd700 1 -- 172.21.15.145:0/3774976290 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e8004390 con 0x55688b806e20 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.244+0000 7ff8deffd700 1 -- 172.21.15.145:0/3774976290 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8019710 con 0x55688b806e20 2023-08-25T04:45:27.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3774976290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 msgr2=0x55688b800700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff8f919c700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 0x7ff8e40a26b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8f419f410 tx=0x7ff8ec009040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009d00 tx=0x7ff8e8011720 comp rx=0 tx=0).stop 2023-08-25T04:45:27.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3774976290 shutdown_connections 2023-08-25T04:45:27.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b801080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b8068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3774976290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b800700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3774976290 >> 172.21.15.145:0/3774976290 conn(0x55688b7750e0 msgr2=0x55688b7fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3774976290 shutdown_connections 2023-08-25T04:45:27.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.245+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3774976290 wait complete. 2023-08-25T04:45:27.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 -- start start 2023-08-25T04:45:27.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 Processor -- start 2023-08-25T04:45:27.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b800c40 0x55688b80ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b80a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b752990 con 0x55688b806500 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b6f0bb0 con 0x55688b800c40 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.246+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b6f08b0 con 0x55688b806e20 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff90e4d1b80 10 client.0 _ll_register_callbacks cb 0x55688b63aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35688/0 (socket says 172.21.15.145:35688) 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3997841244 learned_addr learned my addr 172.21.15.145:0/3997841244 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b800c40 0x55688b80ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f919c700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b80a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3997841244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 msgr2=0x55688b80a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b80a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3997841244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b800c40 msgr2=0x55688b80ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b800c40 0x55688b80ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3997841244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec0090f0 con 0x55688b806500 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0006010 tx=0x7ff8e009a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ad7823f4a1896e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.247+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0096530 con 0x55688b806500 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55688b7d01c0 con 0x55688b806500 2023-08-25T04:45:27.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7ff8e009a8f0 con 0x55688b806500 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e0099fc0 con 0x55688b806500 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 10 client.15630 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff8e8009960 con 0x55688b806500 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55688b7d01c0 con 0x55688b806500 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff8e00ce070 con 0x55688b806500 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:27.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 20 client.15630 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b81a4f0 0x55688b81c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 4 client.15630 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.248+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55688b6f8ba0 con 0x55688b81a4f0 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.249+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b81a4f0 0x55688b81c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.249+0000 7ff8d3fff700 10 client.15630 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.249+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b81a4f0 0x55688b81c8e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ccc621de986dc9 server_cookie=4e7363463928db4d in_seq=0 out_seq=0 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff8d3fff700 1 -- 172.21.15.145:0/3997841244 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55688b6f8ba0 con 0x55688b81a4f0 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff8d3fff700 10 client.15630 handle_command_reply: tid=0 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 client.15630 shutdown 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 msgr2=0x55688b80a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0006010 tx=0x7ff8e009a730 comp rx=0 tx=0).stop 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 shutdown_connections 2023-08-25T04:45:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b81a4f0 0x55688b81c8e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806e20 0x55688b80a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b800c40 0x55688b80ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff8d3fff700 0 client.15630 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3997841244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806500 0x55688b80a040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 >> 172.21.15.145:0/3997841244 conn(0x55688b7750e0 msgr2=0x55688b7ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 shutdown_connections 2023-08-25T04:45:27.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.250+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3997841244 wait complete. 2023-08-25T04:45:27.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.251+0000 7ff90e4d1b80 20 client.15630 trim_cache size 0 max 16384 2023-08-25T04:45:27.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 Processor -- start 2023-08-25T04:45:27.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 -- start start 2023-08-25T04:45:27.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b800700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.283+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b5d4920 con 0x55688b800c40 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b811f60 con 0x55688b806e20 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b7f3690 con 0x55688b806500 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f919c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54236/0 (socket says 172.21.15.145:54236) 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3393041830 learned_addr learned my addr 172.21.15.145:0/3393041830 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f919c700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45620/0 (socket says 172.21.15.145:45620) 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f899b700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b800700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3393041830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 msgr2=0x55688b8119a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3393041830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 msgr2=0x55688b800700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b800700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f3fff700 1 -- 172.21.15.145:0/3393041830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55688b57ede0 con 0x55688b806500 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.284+0000 7ff8f919c700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.285+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009ce0 tx=0x7ff8e00987a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1a5cafd30d3854 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.285+0000 7ff8df7fe700 1 -- 172.21.15.145:0/3393041830 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00ca190 con 0x55688b806500 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.285+0000 7ff8df7fe700 1 -- 172.21.15.145:0/3393041830 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e00d0310 con 0x55688b806500 2023-08-25T04:45:27.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.285+0000 7ff8df7fe700 1 -- 172.21.15.145:0/3393041830 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00a1d20 con 0x55688b806500 2023-08-25T04:45:27.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3393041830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 msgr2=0x55688b809ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009ce0 tx=0x7ff8e00987a0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3393041830 shutdown_connections 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806e20 0x55688b8119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b800c40 0x55688b800700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/3393041830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b806500 0x55688b809ff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.286+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3393041830 >> 172.21.15.145:0/3393041830 conn(0x55688b7750e0 msgr2=0x55688b8098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.287+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3393041830 shutdown_connections 2023-08-25T04:45:27.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.287+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/3393041830 wait complete. 2023-08-25T04:45:27.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.287+0000 7ff90e4d1b80 1 -- start start 2023-08-25T04:45:27.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.287+0000 7ff90e4d1b80 1 Processor -- start 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b80fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806e20 0x55688b810420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b86d4e0 con 0x55688b806e20 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b86d650 con 0x55688b806500 2023-08-25T04:45:27.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55688b86d810 con 0x55688b800c40 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.288+0000 7ff90e4d1b80 10 client.0 _ll_register_callbacks cb 0x55688b63aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54250/0 (socket says 172.21.15.145:54250) 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 -- 172.21.15.145:0/2482392045 learned_addr learned my addr 172.21.15.145:0/2482392045 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 -- 172.21.15.145:0/2482392045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 msgr2=0x55688b80fec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b80fec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 -- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806e20 msgr2=0x55688b810420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806e20 0x55688b810420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.289+0000 7ff8f899b700 1 -- 172.21.15.145:0/2482392045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec01a5a0 con 0x55688b800c40 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.290+0000 7ff8f899b700 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009d00 tx=0x7ff8e80058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cbc533a6d40ae0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.290+0000 7ff8d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004000 con 0x55688b800c40 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7ff8e8004160 con 0x55688b800c40 2023-08-25T04:45:27.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55688b7f38b0 con 0x55688b800c40 2023-08-25T04:45:27.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e8004340 con 0x55688b800c40 2023-08-25T04:45:27.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff90e4d1b80 10 client.25651 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:27.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.291+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff8e8009960 con 0x55688b800c40 2023-08-25T04:45:27.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55688b7f38b0 con 0x55688b800c40 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff8e803d070 con 0x55688b800c40 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 10 client.25651 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 20 client.25651 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b86ed70 0x55688b8711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 4 client.25651 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:45:27.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55688b74b970 con 0x55688b86ed70 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.292+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b86ed70 0x55688b8711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.293+0000 7ff8d3fff700 10 client.25651 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff8f3fff700 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b86ed70 0x55688b8711b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3645516928b33ff7 server_cookie=3a42a564f1e73d41 in_seq=0 out_seq=0 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff8d3fff700 1 -- 172.21.15.145:0/2482392045 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55688b74b970 con 0x55688b86ed70 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff8d3fff700 10 client.25651 handle_command_reply: tid=0 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 client.25651 shutdown 2023-08-25T04:45:27.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 msgr2=0x55688b80f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009d00 tx=0x7ff8e80058e0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 shutdown_connections 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55688b806e20 0x55688b810420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55688b86ed70 0x55688b8711b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55688b800c40 0x55688b80f980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff8d3fff700 0 client.25651 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:27.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 --2- 172.21.15.145:0/2482392045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55688b806500 0x55688b80fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.294+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 >> 172.21.15.145:0/2482392045 conn(0x55688b7750e0 msgr2=0x55688b803f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.295+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 shutdown_connections 2023-08-25T04:45:27.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.295+0000 7ff90e4d1b80 1 -- 172.21.15.145:0/2482392045 wait complete. 2023-08-25T04:45:27.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.295+0000 7ff90e4d1b80 20 client.25651 trim_cache size 0 max 16384 2023-08-25T04:45:27.300 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.295+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 msgr2=0x7ff8e40a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.295+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 0x7ff8e40a26b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8f419f410 tx=0x7ff8ec009040 comp rx=0 tx=0).stop 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 msgr2=0x7ff8f419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005720 comp rx=0 tx=0).stop 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 shutdown_connections 2023-08-25T04:45:27.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4106780 0x7ff8f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff8e40a01f0 0x7ff8e40a26b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8f4067a80 0x7ff8f419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 --2- 172.21.15.145:0/2762722672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4068440 0x7ff8f419ec50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 >> 172.21.15.145:0/2762722672 conn(0x7ff8f40750e0 msgr2=0x7ff8f410b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 shutdown_connections 2023-08-25T04:45:27.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:27.296+0000 7ff8f999d700 1 -- 172.21.15.145:0/2762722672 wait complete. 2023-08-25T04:45:27.571 INFO:teuthology.orchestra.run.smithi145.stderr:2200000 of 50000000 tuples (4%) done (elapsed 5.04 s, remaining 109.52 s) 2023-08-25T04:45:27.828 INFO:teuthology.orchestra.run.smithi145.stderr:2300000 of 50000000 tuples (4%) done (elapsed 5.30 s, remaining 109.87 s) 2023-08-25T04:45:27.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:27 smithi162 ceph-mon[130941]: pgmap v14: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 26 MiB/s wr, 161 op/s 2023-08-25T04:45:27.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:27 smithi183 ceph-mon[131414]: pgmap v14: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 26 MiB/s wr, 161 op/s 2023-08-25T04:45:27.924 INFO:teuthology.orchestra.run.smithi145.stderr:2400000 of 50000000 tuples (4%) done (elapsed 5.39 s, remaining 106.98 s) 2023-08-25T04:45:28.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:27 smithi145 ceph-mon[97681]: pgmap v14: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 26 MiB/s wr, 161 op/s 2023-08-25T04:45:28.204 INFO:teuthology.orchestra.run.smithi145.stderr:2500000 of 50000000 tuples (5%) done (elapsed 5.67 s, remaining 107.81 s) 2023-08-25T04:45:28.320 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-08-25T04:45:28.321 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-08-25T04:45:28.347 INFO:teuthology.orchestra.run.smithi145.stderr:2600000 of 50000000 tuples (5%) done (elapsed 5.82 s, remaining 106.05 s) 2023-08-25T04:45:28.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1350254322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c010ae50 msgr2=0x7fc2c010d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1350254322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c010ae50 0x7fc2c010d310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800b0a0 tx=0x7fc2b800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1350254322 shutdown_connections 2023-08-25T04:45:28.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1350254322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c010ae50 0x7fc2c010d310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1350254322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c0101a30 0x7fc2c0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1350254322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c010a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1350254322 >> 172.21.15.145:0/1350254322 conn(0x7fc2c00fb270 msgr2=0x7fc2c00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1350254322 shutdown_connections 2023-08-25T04:45:28.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.553+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1350254322 wait complete. 2023-08-25T04:45:28.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.554+0000 7fc2c73ff700 1 Processor -- start 2023-08-25T04:45:28.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 -- start start 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0101a30 0x7fc2c0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c010ae50 0x7fc2c019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019dd40 con 0x7fc2c0102350 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019deb0 con 0x7fc2c010ae50 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.555+0000 7fc2c73ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c019e070 con 0x7fc2c0101a30 2023-08-25T04:45:28.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c599c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c010ae50 0x7fc2c019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35734/0 (socket says 172.21.15.145:35734) 2023-08-25T04:45:28.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 -- 172.21.15.145:0/1521250797 learned_addr learned my addr 172.21.15.145:0/1521250797 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 -- 172.21.15.145:0/1521250797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0101a30 msgr2=0x7fc2c0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0101a30 0x7fc2c0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 -- 172.21.15.145:0/1521250797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c010ae50 msgr2=0x7fc2c019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c010ae50 0x7fc2c019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.556+0000 7fc2c499a700 1 -- 172.21.15.145:0/1521250797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8009d00 con 0x7fc2c0102350 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000e9f0 tx=0x7fc2b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2b67fc700 1 -- 172.21.15.145:0/1521250797 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000eee0 con 0x7fc2c0102350 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2b67fc700 1 -- 172.21.15.145:0/1521250797 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b000cbd0 con 0x7fc2c0102350 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2b67fc700 1 -- 172.21.15.145:0/1521250797 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0018680 con 0x7fc2c0102350 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c019e350 con 0x7fc2c0102350 2023-08-25T04:45:28.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.557+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c01a5b90 con 0x7fc2c0102350 2023-08-25T04:45:28.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.562+0000 7fc2b67fc700 1 -- 172.21.15.145:0/1521250797 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7fc2b0010430 con 0x7fc2c0102350 2023-08-25T04:45:28.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2b67fc700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 0x7fc2ac0a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 Processor -- start 2023-08-25T04:45:28.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2b67fc700 1 -- 172.21.15.145:0/1521250797 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc2b0014070 con 0x7fc2c0102350 2023-08-25T04:45:28.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 -- start start 2023-08-25T04:45:28.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2c519b700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 0x7fc2ac0a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e432237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223d20 0x557e4321d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e43185830 con 0x557e43223d20 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e43176800 con 0x557e4321dcc0 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.564+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e43174120 con 0x557e43223400 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c499a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223d20 0x557e4321d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e432237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45670/0 (socket says 172.21.15.145:45670) 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 -- 172.21.15.145:0/2570824615 learned_addr learned my addr 172.21.15.145:0/2570824615 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 -- 172.21.15.145:0/2570824615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 msgr2=0x557e432237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e432237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 -- 172.21.15.145:0/2570824615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223d20 msgr2=0x557e4321d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223d20 0x557e4321d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 -- 172.21.15.145:0/2570824615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e42fef6b0 con 0x557e4321dcc0 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c519b700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 0x7fc2ac0a2690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005950 tx=0x7fc2bc012780 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.565+0000 7fc2c599c700 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8015000 tx=0x7fc2b8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83932166d2a16da2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2ab7fe700 1 -- 172.21.15.145:0/2570824615 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800e040 con 0x557e4321dcc0 2023-08-25T04:45:28.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2ab7fe700 1 -- 172.21.15.145:0/2570824615 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b8003a00 con 0x557e4321dcc0 2023-08-25T04:45:28.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2ab7fe700 1 -- 172.21.15.145:0/2570824615 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b801c440 con 0x557e4321dcc0 2023-08-25T04:45:28.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570824615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 msgr2=0x557e4321e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8015000 tx=0x7fc2b8009c20 comp rx=0 tx=0).stop 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570824615 shutdown_connections 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.566+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e4321e120 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e432237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570824615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223d20 0x557e4321d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570824615 >> 172.21.15.145:0/2570824615 conn(0x557e43192160 msgr2=0x557e43218490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570824615 shutdown_connections 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570824615 wait complete. 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 -- start start 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.567+0000 7fc2dad2cb80 1 Processor -- start 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223400 0x557e43226f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e432275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4316fa10 con 0x557e43223400 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4310dc30 con 0x557e4321dcc0 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4310d930 con 0x557e43223d20 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2dad2cb80 10 client.0 _ll_register_callbacks cb 0x557e43057b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:45684/0 (socket says 172.21.15.145:45684) 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 -- 172.21.15.145:0/2555328200 learned_addr learned my addr 172.21.15.145:0/2555328200 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c599c700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e432275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223400 0x557e43226f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 -- 172.21.15.145:0/2555328200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 msgr2=0x557e432275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e432275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 -- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223400 msgr2=0x557e43226f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223400 0x557e43226f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.568+0000 7fc2c519b700 1 -- 172.21.15.145:0/2555328200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c010dfd0 con 0x557e4321dcc0 2023-08-25T04:45:28.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2c519b700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc01d8d0 tx=0x7fc2bc018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd229b40a188074 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc01e660 con 0x557e4321dcc0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e431ed240 con 0x557e4321dcc0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc2bc022070 con 0x557e4321dcc0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2bc01e7c0 con 0x557e4321dcc0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2dad2cb80 10 client.25646 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e42f9be60 con 0x557e4321dcc0 2023-08-25T04:45:28.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e431ed240 con 0x557e4321dcc0 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.569+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2bc010af0 con 0x557e4321dcc0 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 10 client.25646 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 20 client.25646 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e43237760 0x557e43239b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 4 client.25646 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:28.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e43115c20 con 0x557e43237760 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e43237760 0x557e43239b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e43237760 0x557e43239b80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22071887779ad239 server_cookie=918e5172e6d3c517 in_seq=0 out_seq=0 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.570+0000 7fc2a8ff9700 10 client.25646 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/2555328200 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e43115c20 con 0x557e43237760 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2a8ff9700 10 client.25646 handle_command_reply: tid=0 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 client.25646 shutdown 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 msgr2=0x557e43229c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc01d8d0 tx=0x7fc2bc018000 comp rx=0 tx=0).stop 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 shutdown_connections 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e43237760 0x557e43239b80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e432275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e4321dcc0 0x557e43229c80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2555328200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e43223400 0x557e43226f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2a8ff9700 0 client.25646 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 >> 172.21.15.145:0/2555328200 conn(0x557e43192160 msgr2=0x557e43212bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.571+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 shutdown_connections 2023-08-25T04:45:28.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.572+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2555328200 wait complete. 2023-08-25T04:45:28.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.572+0000 7fc2dad2cb80 20 client.25646 trim_cache size 0 max 16384 2023-08-25T04:45:28.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.589+0000 7fc2dad2cb80 1 Processor -- start 2023-08-25T04:45:28.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.589+0000 7fc2dad2cb80 1 -- start start 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e43233c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223d20 0x557e43229820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42ff19a0 con 0x557e4321dcc0 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e43221c50 con 0x557e43223d20 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4321a210 con 0x557e43223400 2023-08-25T04:45:28.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e43233c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54308/0 (socket says 172.21.15.145:54308) 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 -- 172.21.15.145:0/2570772700 learned_addr learned my addr 172.21.15.145:0/2570772700 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c599c700 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223d20 0x557e43229820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 -- 172.21.15.145:0/2570772700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223d20 msgr2=0x557e43229820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223d20 0x557e43229820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 -- 172.21.15.145:0/2570772700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 msgr2=0x557e43233c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e43233c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.590+0000 7fc2c499a700 1 -- 172.21.15.145:0/2570772700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e42f84ce0 con 0x557e43223400 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.591+0000 7fc2c499a700 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c5b0 tx=0x7fc2b00b8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba8c6c19c80812b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.591+0000 7fc2abfff700 1 -- 172.21.15.145:0/2570772700 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0096f30 con 0x557e43223400 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.591+0000 7fc2abfff700 1 -- 172.21.15.145:0/2570772700 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b0097190 con 0x557e43223400 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.591+0000 7fc2abfff700 1 -- 172.21.15.145:0/2570772700 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00c9190 con 0x557e43223400 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570772700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 msgr2=0x557e43222770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c5b0 tx=0x7fc2b00b8040 comp rx=0 tx=0).stop 2023-08-25T04:45:28.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570772700 shutdown_connections 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223d20 0x557e43229820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e43233c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/2570772700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223400 0x557e43222770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570772700 >> 172.21.15.145:0/2570772700 conn(0x557e43192160 msgr2=0x557e43222060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570772700 shutdown_connections 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/2570772700 wait complete. 2023-08-25T04:45:28.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 -- start start 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.592+0000 7fc2dad2cb80 1 Processor -- start 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223400 0x557e4321fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e4328b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e43220460 con 0x557e4321dcc0 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e432205d0 con 0x557e43223400 2023-08-25T04:45:28.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4328bb20 con 0x557e43223d20 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2dad2cb80 10 client.0 _ll_register_callbacks cb 0x7fc2bc01d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35760/0 (socket says 172.21.15.145:35760) 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 -- 172.21.15.145:0/1135347760 learned_addr learned my addr 172.21.15.145:0/1135347760 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223400 0x557e4321fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c599c700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e4328b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 -- 172.21.15.145:0/1135347760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 msgr2=0x557e4328b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e4328b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 -- 172.21.15.145:0/1135347760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223400 msgr2=0x557e4321fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223400 0x557e4321fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.593+0000 7fc2c519b700 1 -- 172.21.15.145:0/1135347760 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c01a5b90 con 0x557e4321dcc0 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2c519b700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e000 tx=0x7fc2bc026650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70904eca620babb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc009d10 con 0x557e4321dcc0 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc2bc022040 con 0x557e4321dcc0 2023-08-25T04:45:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e43210930 con 0x557e4321dcc0 2023-08-25T04:45:28.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2bc0269b0 con 0x557e4321dcc0 2023-08-25T04:45:28.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2dad2cb80 10 client.15651 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:28.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.594+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e42fc22d0 con 0x557e4321dcc0 2023-08-25T04:45:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e43210930 con 0x557e4321dcc0 2023-08-25T04:45:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2bc005020 con 0x557e4321dcc0 2023-08-25T04:45:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 10 client.15651 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 20 client.15651 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e4328d270 0x557e4328f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 4 client.15651 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e431689f0 con 0x557e4328d270 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.595+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e4328d270 0x557e4328f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.596+0000 7fc2c499a700 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e4328d270 0x557e4328f690 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6cd293e936c5463 server_cookie=9a0fc68d9b2c789f in_seq=0 out_seq=0 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.596+0000 7fc2a8ff9700 10 client.15651 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2a8ff9700 1 -- 172.21.15.145:0/1135347760 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e431689f0 con 0x557e4328d270 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2a8ff9700 10 client.15651 handle_command_reply: tid=0 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 client.15651 shutdown 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 msgr2=0x557e4321f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e000 tx=0x7fc2bc026650 comp rx=0 tx=0).stop 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 shutdown_connections 2023-08-25T04:45:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557e4328d270 0x557e4328f690 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.597+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e43223d20 0x557e4328b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557e4321dcc0 0x557e4321f750 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 1 --2- 172.21.15.145:0/1135347760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557e43223400 0x557e4321fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 >> 172.21.15.145:0/1135347760 conn(0x557e43192160 msgr2=0x557e43212330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2a8ff9700 0 client.15651 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 shutdown_connections 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 1 -- 172.21.15.145:0/1135347760 wait complete. 2023-08-25T04:45:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.598+0000 7fc2dad2cb80 20 client.15651 trim_cache size 0 max 16384 2023-08-25T04:45:28.605 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:45:28.605 INFO:teuthology.orchestra.run.smithi145.stdout: "return_code": 0, 2023-08-25T04:45:28.605 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_tag": "67ef9032-8d65-4588-b5c1-86f6dde4132c", 2023-08-25T04:45:28.605 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": "asynchronous" 2023-08-25T04:45:28.605 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:45:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 msgr2=0x7fc2ac0a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 0x7fc2ac0a2690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005950 tx=0x7fc2bc012780 comp rx=0 tx=0).stop 2023-08-25T04:45:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 msgr2=0x7fc2c01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000e9f0 tx=0x7fc2b000ed00 comp rx=0 tx=0).stop 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 shutdown_connections 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc2ac0a01d0 0x7fc2ac0a2690 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c010ae50 0x7fc2c019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0101a30 0x7fc2c0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 --2- 172.21.15.145:0/1521250797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2c0102350 0x7fc2c01992a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 >> 172.21.15.145:0/1521250797 conn(0x7fc2c00fb270 msgr2=0x7fc2c010d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 shutdown_connections 2023-08-25T04:45:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.599+0000 7fc2c73ff700 1 -- 172.21.15.145:0/1521250797 wait complete. 2023-08-25T04:45:28.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:45:28.635 INFO:teuthology.orchestra.run.smithi145.stderr:2700000 of 50000000 tuples (5%) done (elapsed 6.11 s, remaining 106.96 s) 2023-08-25T04:45:28.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 -- 172.21.15.145:0/645993117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680010ae50 msgr2=0x7f680010d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 --2- 172.21.15.145:0/645993117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680010ae50 0x7f680010d310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b0a0 tx=0x7f67f800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 -- 172.21.15.145:0/645993117 shutdown_connections 2023-08-25T04:45:28.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 --2- 172.21.15.145:0/645993117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680010ae50 0x7f680010d310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 --2- 172.21.15.145:0/645993117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 --2- 172.21.15.145:0/645993117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6800102350 0x7f680010a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 -- 172.21.15.145:0/645993117 >> 172.21.15.145:0/645993117 conn(0x7f68000fb270 msgr2=0x7f68000fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 -- 172.21.15.145:0/645993117 shutdown_connections 2023-08-25T04:45:28.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.925+0000 7f68065ff700 1 -- 172.21.15.145:0/645993117 wait complete. 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.926+0000 7f68065ff700 1 Processor -- start 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.926+0000 7f68065ff700 1 -- start start 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 0x7f6800102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680010ae50 0x7f68001032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001a9400 con 0x7f680010ae50 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001a9570 con 0x7f6800101a30 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f68065ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001a9730 con 0x7f6800102350 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 0x7f6800102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 0x7f6800102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54326/0 (socket says 172.21.15.145:54326) 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67ff7fe700 1 -- 172.21.15.145:0/3807331454 learned_addr learned my addr 172.21.15.145:0/3807331454 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f6804b9c700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680010ae50 0x7f68001032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 -- 172.21.15.145:0/3807331454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 msgr2=0x7f6800102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 0x7f6800102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 -- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680010ae50 msgr2=0x7f68001032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680010ae50 0x7f68001032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.927+0000 7f67fffff700 1 -- 172.21.15.145:0/3807331454 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009d00 con 0x7f6800101a30 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f67fffff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800102870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.931 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f67fd7fa700 1 -- 172.21.15.145:0/3807331454 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400c9c0 con 0x7f6800101a30 2023-08-25T04:45:28.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f67fd7fa700 1 -- 172.21.15.145:0/3807331454 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f4010430 con 0x7f6800101a30 2023-08-25T04:45:28.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f67fd7fa700 1 -- 172.21.15.145:0/3807331454 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018680 con 0x7f6800101a30 2023-08-25T04:45:28.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68001a9a10 con 0x7f6800101a30 2023-08-25T04:45:28.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.928+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68001a9f20 con 0x7f6800101a30 2023-08-25T04:45:28.932 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.931+0000 7f67fd7fa700 1 -- 172.21.15.145:0/3807331454 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f67f4010860 con 0x7f6800101a30 2023-08-25T04:45:28.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.933+0000 7f67fd7fa700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 0x7f67e80a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.933 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.933+0000 7f67fd7fa700 1 -- 172.21.15.145:0/3807331454 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f67f4014070 con 0x7f6800101a30 2023-08-25T04:45:28.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.933+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 0x7f67e80a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.934+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 0x7f67e80a2690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6800104060 tx=0x7f67f0009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.934 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.934+0000 7f6819f35b80 1 Processor -- start 2023-08-25T04:45:28.935 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.934+0000 7f6819f35b80 1 -- start start 2023-08-25T04:45:28.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 0x562501845700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625017ad7b0 con 0x56250184b380 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250179e780 con 0x56250184bca0 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6819f35b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250179c0a0 con 0x562501845c40 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6804b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6804b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54330/0 (socket says 172.21.15.145:54330) 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f6804b9c700 1 -- 172.21.15.145:0/1485770494 learned_addr learned my addr 172.21.15.145:0/1485770494 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 0x562501845700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.935+0000 7f67fffff700 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 -- 172.21.15.145:0/1485770494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 msgr2=0x5625018460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 -- 172.21.15.145:0/1485770494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 msgr2=0x56250184b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 -- 172.21.15.145:0/1485770494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562501617630 con 0x56250184bca0 2023-08-25T04:45:28.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 0x562501845700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67f0011ab0 tx=0x7f67f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9c8b6fc09bb1e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67e67fc700 1 -- 172.21.15.145:0/1485770494 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f000ec20 con 0x56250184bca0 2023-08-25T04:45:28.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67e67fc700 1 -- 172.21.15.145:0/1485770494 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f000ed80 con 0x56250184bca0 2023-08-25T04:45:28.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.936+0000 7f67e67fc700 1 -- 172.21.15.145:0/1485770494 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f00056c0 con 0x56250184bca0 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.937+0000 7f6819f35b80 1 -- 172.21.15.145:0/1485770494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 msgr2=0x562501845700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.937+0000 7f6819f35b80 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 0x562501845700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67f0011ab0 tx=0x7f67f0014000 comp rx=0 tx=0).stop 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 -- 172.21.15.145:0/1485770494 shutdown_connections 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 --2- 172.21.15.145:0/1485770494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184bca0 0x562501845700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 -- 172.21.15.145:0/1485770494 >> 172.21.15.145:0/1485770494 conn(0x5625017ba0e0 msgr2=0x562501840410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 -- 172.21.15.145:0/1485770494 shutdown_connections 2023-08-25T04:45:28.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 -- 172.21.15.145:0/1485770494 wait complete. 2023-08-25T04:45:28.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.938+0000 7f6819f35b80 1 -- start start 2023-08-25T04:45:28.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 Processor -- start 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x56250184f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562501797990 con 0x56250184b380 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562501735bb0 con 0x562501845c40 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625017358b0 con 0x56250184faf0 2023-08-25T04:45:28.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6819f35b80 10 client.0 _ll_register_callbacks cb 0x56250167fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6804b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6804b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54342/0 (socket says 172.21.15.145:54342) 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f6804b9c700 1 -- 172.21.15.145:0/2825085697 learned_addr learned my addr 172.21.15.145:0/2825085697 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.939+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f67fffff700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x56250184f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 -- 172.21.15.145:0/2825085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 msgr2=0x56250184f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x56250184f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 -- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 msgr2=0x56250184f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 -- 172.21.15.145:0/2825085697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f0017070 con 0x56250184faf0 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6804b9c700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67f80062a0 tx=0x7f67f800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fde88505616642d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f67df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:28.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e040 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625018151c0 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.940+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f67f80074e0 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67f80076a0 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 10 client.25678 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5625015c3de0 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625018151c0 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f67f8024030 con 0x56250184faf0 2023-08-25T04:45:28.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 20 client.25678 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56250185fc90 0x5625018620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 4 client.25678 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.941+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56250173dba0 con 0x56250185fc90 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.942+0000 7f67fffff700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56250185fc90 0x5625018620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.942+0000 7f67fffff700 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56250185fc90 0x5625018620b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91db3eff5ad4292 server_cookie=27347adab009fe2c in_seq=0 out_seq=0 2023-08-25T04:45:28.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.942+0000 7f67df7fe700 10 client.25678 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f67df7fe700 1 -- 172.21.15.145:0/2825085697 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56250173dba0 con 0x56250185fc90 2023-08-25T04:45:28.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f67df7fe700 10 client.25678 handle_command_reply: tid=0 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 client.25678 shutdown 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 msgr2=0x56250185b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67f80062a0 tx=0x7f67f800bf90 comp rx=0 tx=0).stop 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 shutdown_connections 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56250185fc90 0x5625018620b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185b850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x56250184f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f67df7fe700 0 client.25678 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 --2- 172.21.15.145:0/2825085697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 >> 172.21.15.145:0/2825085697 conn(0x5625017ba0e0 msgr2=0x56250183a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 shutdown_connections 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.943+0000 7f6819f35b80 1 -- 172.21.15.145:0/2825085697 wait complete. 2023-08-25T04:45:28.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.944+0000 7f6819f35b80 20 client.25678 trim_cache size 0 max 16384 2023-08-25T04:45:28.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 Processor -- start 2023-08-25T04:45:28.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 -- start start 2023-08-25T04:45:28.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x562501845700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562501619920 con 0x56250184b380 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625018415e0 con 0x562501845c40 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.962+0000 7f6819f35b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x562501850680 con 0x56250184faf0 2023-08-25T04:45:28.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:35798/0 (socket says 172.21.15.145:35798) 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67ff7fe700 1 -- 172.21.15.145:0/4291974719 learned_addr learned my addr 172.21.15.145:0/4291974719 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67fffff700 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x562501845700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 -- 172.21.15.145:0/4291974719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 msgr2=0x562501845700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x562501845700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 -- 172.21.15.145:0/4291974719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 msgr2=0x56250184e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 -- 172.21.15.145:0/4291974719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625015acc60 con 0x56250184faf0 2023-08-25T04:45:28.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67fffff700 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x562501845700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:45:28.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f6804b9c700 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185aef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67f801c410 tx=0x7f67f8006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e2b5a14a7d410f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.963+0000 7f67e6ffd700 1 -- 172.21.15.145:0/4291974719 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e040 con 0x56250184faf0 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.964+0000 7f67e6ffd700 1 -- 172.21.15.145:0/4291974719 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f801ce50 con 0x56250184faf0 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.964+0000 7f67e6ffd700 1 -- 172.21.15.145:0/4291974719 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8026660 con 0x56250184faf0 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- 172.21.15.145:0/4291974719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 msgr2=0x56250185aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185aef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67f801c410 tx=0x7f67f8006000 comp rx=0 tx=0).stop 2023-08-25T04:45:28.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- 172.21.15.145:0/4291974719 shutdown_connections 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56250184faf0 0x56250185aef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562501845c40 0x562501845700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 --2- 172.21.15.145:0/4291974719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184b380 0x56250184e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- 172.21.15.145:0/4291974719 >> 172.21.15.145:0/4291974719 conn(0x5625017ba0e0 msgr2=0x562501849150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- 172.21.15.145:0/4291974719 shutdown_connections 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- 172.21.15.145:0/4291974719 wait complete. 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.965+0000 7f6819f35b80 1 -- start start 2023-08-25T04:45:28.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 Processor -- start 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184b380 0x562501854a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184faf0 0x5625018b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625018551b0 con 0x56250184faf0 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625018b3010 con 0x56250184b380 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625018b31d0 con 0x562501845c40 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.966+0000 7f6819f35b80 10 client.0 _ll_register_callbacks cb 0x56250167fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:28.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54356/0 (socket says 172.21.15.145:54356) 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 -- 172.21.15.145:0/244158040 learned_addr learned my addr 172.21.15.145:0/244158040 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f6804b9c700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184faf0 0x5625018b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184b380 0x562501854a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 -- 172.21.15.145:0/244158040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184b380 msgr2=0x562501854a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184b380 0x562501854a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 -- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184faf0 msgr2=0x5625018b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:28.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184faf0 0x5625018b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 -- 172.21.15.145:0/244158040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625015c3de0 con 0x562501845c40 2023-08-25T04:45:28.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67fffff700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67f4098d50 tx=0x7f67f40b8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c0908eeba304717 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:28.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.967+0000 7f67df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:28.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f409cbf0 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f67f40a0bd0 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625018388b0 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67f40c9160 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 10 client.25684 fetch_fsmap learned FSMap version 23 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5625015ea250 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625018388b0 con 0x562501845c40 2023-08-25T04:45:28.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f67f4099020 con 0x562501845c40 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 23 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 20 client.25684 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5625018b4930 0x5625018b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 4 client.25684 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.968+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562501790970 con 0x5625018b4930 2023-08-25T04:45:28.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5625018b4930 0x5625018b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f67df7fe700 10 client.25684 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f67ff7fe700 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5625018b4930 0x5625018b6d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c59905bc6c49664 server_cookie=535fe7852d335790 in_seq=0 out_seq=0 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f67df7fe700 1 -- 172.21.15.145:0/244158040 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562501790970 con 0x5625018b4930 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f67df7fe700 10 client.25684 handle_command_reply: tid=0 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f6819f35b80 1 client.25684 shutdown 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 msgr2=0x5625018544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.969+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67f4098d50 tx=0x7f67f40b8040 comp rx=0 tx=0).stop 2023-08-25T04:45:28.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 shutdown_connections 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56250184faf0 0x5625018b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5625018b4930 0x5625018b6d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562501845c40 0x5625018544d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 --2- 172.21.15.145:0/244158040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56250184b380 0x562501854a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f67df7fe700 0 client.25684 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 >> 172.21.15.145:0/244158040 conn(0x5625017ba0e0 msgr2=0x56250183a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 shutdown_connections 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 1 -- 172.21.15.145:0/244158040 wait complete. 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f6819f35b80 20 client.25684 trim_cache size 0 max 16384 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "IDLE (waiting for more scrubs)", 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:45:28.978 INFO:teuthology.orchestra.run.smithi145.stdout: "67ef9032-8d65-4588-b5c1-86f6dde4132c": { 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "67ef9032-8d65-4588-b5c1-86f6dde4132c", 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:45:28.979 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:45:28.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 msgr2=0x7f67e80a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.970+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 0x7f67e80a2690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6800104060 tx=0x7f67f0009260 comp rx=0 tx=0).stop 2023-08-25T04:45:28.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 msgr2=0x7f6800102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:28.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800102870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400ed00 comp rx=0 tx=0).stop 2023-08-25T04:45:28.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 shutdown_connections 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f680010ae50 0x7f68001032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f67e80a01d0 0x7f67e80a2690 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6800101a30 0x7f6800102870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 --2- 172.21.15.145:0/3807331454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800102350 0x7f6800102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 >> 172.21.15.145:0/3807331454 conn(0x7f68000fb270 msgr2=0x7f680010cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 shutdown_connections 2023-08-25T04:45:28.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:28.971+0000 7f68065ff700 1 -- 172.21.15.145:0/3807331454 wait complete. 2023-08-25T04:45:28.983 INFO:teuthology.orchestra.run.smithi145.stderr:2800000 of 50000000 tuples (5%) done (elapsed 6.45 s, remaining 108.78 s) 2023-08-25T04:45:28.989 INFO:tasks.cephfs.filesystem:scrub status for tag:67ef9032-8d65-4588-b5c1-86f6dde4132c - {'path': '/', 'tag': '67ef9032-8d65-4588-b5c1-86f6dde4132c', 'options': 'recursive,force'} 2023-08-25T04:45:29.126 INFO:teuthology.orchestra.run.smithi145.stderr:2900000 of 50000000 tuples (5%) done (elapsed 6.60 s, remaining 107.13 s) 2023-08-25T04:45:29.337 INFO:teuthology.orchestra.run.smithi145.stderr:3000000 of 50000000 tuples (6%) done (elapsed 6.81 s, remaining 106.63 s) 2023-08-25T04:45:29.492 INFO:teuthology.orchestra.run.smithi145.stderr:3100000 of 50000000 tuples (6%) done (elapsed 6.96 s, remaining 105.32 s) 2023-08-25T04:45:29.762 INFO:teuthology.orchestra.run.smithi145.stderr:3200000 of 50000000 tuples (6%) done (elapsed 7.23 s, remaining 105.77 s) 2023-08-25T04:45:29.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:29 smithi162 ceph-mon[130941]: pgmap v15: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 25 MiB/s wr, 140 op/s 2023-08-25T04:45:29.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:29 smithi162 ceph-mon[130941]: scrub queued for path: / 2023-08-25T04:45:29.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:29 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:29.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:29 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:45:29.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:29 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:29.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:29 smithi183 ceph-mon[131414]: pgmap v15: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 25 MiB/s wr, 140 op/s 2023-08-25T04:45:29.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:29 smithi183 ceph-mon[131414]: scrub queued for path: / 2023-08-25T04:45:29.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:29 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:29 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:45:29.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:29 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:30.068 INFO:teuthology.orchestra.run.smithi145.stderr:3300000 of 50000000 tuples (6%) done (elapsed 7.54 s, remaining 106.67 s) 2023-08-25T04:45:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:29 smithi145 ceph-mon[97681]: pgmap v15: 129 pgs: 129 active+clean; 171 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 25 MiB/s wr, 140 op/s 2023-08-25T04:45:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:29 smithi145 ceph-mon[97681]: scrub queued for path: / 2023-08-25T04:45:30.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:29 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:30.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:29 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:45:30.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:29 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:45:30.211 INFO:teuthology.orchestra.run.smithi145.stderr:3400000 of 50000000 tuples (6%) done (elapsed 7.68 s, remaining 105.27 s) 2023-08-25T04:45:30.489 INFO:teuthology.orchestra.run.smithi145.stderr:3500000 of 50000000 tuples (7%) done (elapsed 7.96 s, remaining 105.74 s) 2023-08-25T04:45:30.796 INFO:teuthology.orchestra.run.smithi145.stderr:3600000 of 50000000 tuples (7%) done (elapsed 8.27 s, remaining 106.54 s) 2023-08-25T04:45:30.905 INFO:teuthology.orchestra.run.smithi145.stderr:3700000 of 50000000 tuples (7%) done (elapsed 8.38 s, remaining 104.80 s) 2023-08-25T04:45:30.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:30 smithi162 ceph-mon[130941]: scrub summary: idle 2023-08-25T04:45:30.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:30 smithi183 ceph-mon[131414]: scrub summary: idle 2023-08-25T04:45:31.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:30 smithi145 ceph-mon[97681]: scrub summary: idle 2023-08-25T04:45:31.206 INFO:teuthology.orchestra.run.smithi145.stderr:3800000 of 50000000 tuples (7%) done (elapsed 8.68 s, remaining 105.49 s) 2023-08-25T04:45:31.490 INFO:teuthology.orchestra.run.smithi145.stderr:3900000 of 50000000 tuples (7%) done (elapsed 8.96 s, remaining 105.92 s) 2023-08-25T04:45:31.636 INFO:teuthology.orchestra.run.smithi145.stderr:4000000 of 50000000 tuples (8%) done (elapsed 9.11 s, remaining 104.72 s) 2023-08-25T04:45:31.898 INFO:teuthology.orchestra.run.smithi145.stderr:4100000 of 50000000 tuples (8%) done (elapsed 9.37 s, remaining 104.87 s) 2023-08-25T04:45:31.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:31 smithi162 ceph-mon[130941]: pgmap v16: 129 pgs: 129 active+clean; 295 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 45 MiB/s wr, 147 op/s 2023-08-25T04:45:31.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:31 smithi183 ceph-mon[131414]: pgmap v16: 129 pgs: 129 active+clean; 295 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 45 MiB/s wr, 147 op/s 2023-08-25T04:45:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:31 smithi145 ceph-mon[97681]: pgmap v16: 129 pgs: 129 active+clean; 295 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 45 MiB/s wr, 147 op/s 2023-08-25T04:45:32.194 INFO:teuthology.orchestra.run.smithi145.stderr:4200000 of 50000000 tuples (8%) done (elapsed 9.66 s, remaining 105.38 s) 2023-08-25T04:45:32.325 INFO:teuthology.orchestra.run.smithi145.stderr:4300000 of 50000000 tuples (8%) done (elapsed 9.79 s, remaining 104.10 s) 2023-08-25T04:45:32.611 INFO:teuthology.orchestra.run.smithi145.stderr:4400000 of 50000000 tuples (8%) done (elapsed 10.08 s, remaining 104.48 s) 2023-08-25T04:45:32.779 INFO:teuthology.orchestra.run.smithi145.stderr:4500000 of 50000000 tuples (9%) done (elapsed 10.25 s, remaining 103.63 s) 2023-08-25T04:45:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:4600000 of 50000000 tuples (9%) done (elapsed 10.52 s, remaining 103.78 s) 2023-08-25T04:45:33.314 INFO:teuthology.orchestra.run.smithi145.stderr:4700000 of 50000000 tuples (9%) done (elapsed 10.78 s, remaining 103.94 s) 2023-08-25T04:45:33.490 INFO:teuthology.orchestra.run.smithi145.stderr:4800000 of 50000000 tuples (9%) done (elapsed 10.96 s, remaining 103.20 s) 2023-08-25T04:45:33.730 INFO:teuthology.orchestra.run.smithi145.stderr:4900000 of 50000000 tuples (9%) done (elapsed 11.20 s, remaining 103.09 s) 2023-08-25T04:45:33.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:33 smithi162 ceph-mon[130941]: pgmap v17: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 61 MiB/s wr, 162 op/s 2023-08-25T04:45:33.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:33 smithi183 ceph-mon[131414]: pgmap v17: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 61 MiB/s wr, 162 op/s 2023-08-25T04:45:34.038 INFO:teuthology.orchestra.run.smithi145.stderr:5000000 of 50000000 tuples (10%) done (elapsed 11.51 s, remaining 103.57 s) 2023-08-25T04:45:34.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:33 smithi145 ceph-mon[97681]: pgmap v17: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 61 MiB/s wr, 162 op/s 2023-08-25T04:45:34.207 INFO:teuthology.orchestra.run.smithi145.stderr:5100000 of 50000000 tuples (10%) done (elapsed 11.68 s, remaining 102.80 s) 2023-08-25T04:45:34.472 INFO:teuthology.orchestra.run.smithi145.stderr:5200000 of 50000000 tuples (10%) done (elapsed 11.94 s, remaining 102.89 s) 2023-08-25T04:45:34.759 INFO:teuthology.orchestra.run.smithi145.stderr:5300000 of 50000000 tuples (10%) done (elapsed 12.23 s, remaining 103.14 s) 2023-08-25T04:45:34.923 INFO:teuthology.orchestra.run.smithi145.stderr:5400000 of 50000000 tuples (10%) done (elapsed 12.39 s, remaining 102.36 s) 2023-08-25T04:45:35.191 INFO:teuthology.orchestra.run.smithi145.stderr:5500000 of 50000000 tuples (11%) done (elapsed 12.66 s, remaining 102.44 s) 2023-08-25T04:45:35.500 INFO:teuthology.orchestra.run.smithi145.stderr:5600000 of 50000000 tuples (11%) done (elapsed 12.97 s, remaining 102.83 s) 2023-08-25T04:45:35.623 INFO:teuthology.orchestra.run.smithi145.stderr:5700000 of 50000000 tuples (11%) done (elapsed 13.09 s, remaining 101.76 s) 2023-08-25T04:45:35.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:35 smithi162 ceph-mon[130941]: pgmap v18: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 61 MiB/s wr, 158 op/s 2023-08-25T04:45:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:35 smithi183 ceph-mon[131414]: pgmap v18: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 61 MiB/s wr, 158 op/s 2023-08-25T04:45:35.950 INFO:teuthology.orchestra.run.smithi145.stderr:5800000 of 50000000 tuples (11%) done (elapsed 13.42 s, remaining 102.27 s) 2023-08-25T04:45:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:35 smithi145 ceph-mon[97681]: pgmap v18: 129 pgs: 129 active+clean; 395 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 61 MiB/s wr, 158 op/s 2023-08-25T04:45:36.530 INFO:teuthology.orchestra.run.smithi145.stderr:5900000 of 50000000 tuples (11%) done (elapsed 14.00 s, remaining 104.65 s) 2023-08-25T04:45:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:6000000 of 50000000 tuples (12%) done (elapsed 14.16 s, remaining 103.87 s) 2023-08-25T04:45:36.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:36 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:36.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:36 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:37.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:36 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:37.527 INFO:teuthology.orchestra.run.smithi145.stderr:6100000 of 50000000 tuples (12%) done (elapsed 15.00 s, remaining 107.93 s) 2023-08-25T04:45:37.816 INFO:teuthology.orchestra.run.smithi145.stderr:6200000 of 50000000 tuples (12%) done (elapsed 15.29 s, remaining 107.98 s) 2023-08-25T04:45:37.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:37 smithi162 ceph-mon[130941]: pgmap v19: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 98 MiB/s wr, 184 op/s 2023-08-25T04:45:37.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:37 smithi183 ceph-mon[131414]: pgmap v19: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 98 MiB/s wr, 184 op/s 2023-08-25T04:45:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:37 smithi145 ceph-mon[97681]: pgmap v19: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 98 MiB/s wr, 184 op/s 2023-08-25T04:45:38.236 INFO:teuthology.orchestra.run.smithi145.stderr:6300000 of 50000000 tuples (12%) done (elapsed 15.71 s, remaining 108.95 s) 2023-08-25T04:45:38.502 INFO:teuthology.orchestra.run.smithi145.stderr:6400000 of 50000000 tuples (12%) done (elapsed 15.97 s, remaining 108.81 s) 2023-08-25T04:45:38.679 INFO:teuthology.orchestra.run.smithi145.stderr:6500000 of 50000000 tuples (13%) done (elapsed 16.15 s, remaining 108.07 s) 2023-08-25T04:45:38.982 INFO:teuthology.orchestra.run.smithi145.stderr:6600000 of 50000000 tuples (13%) done (elapsed 16.45 s, remaining 108.18 s) 2023-08-25T04:45:39.245 INFO:teuthology.orchestra.run.smithi145.stderr:6700000 of 50000000 tuples (13%) done (elapsed 16.71 s, remaining 108.02 s) 2023-08-25T04:45:39.412 INFO:teuthology.orchestra.run.smithi145.stderr:6800000 of 50000000 tuples (13%) done (elapsed 16.88 s, remaining 107.25 s) 2023-08-25T04:45:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:6900000 of 50000000 tuples (13%) done (elapsed 17.10 s, remaining 106.83 s) 2023-08-25T04:45:39.793 INFO:teuthology.orchestra.run.smithi145.stderr:7000000 of 50000000 tuples (14%) done (elapsed 17.26 s, remaining 106.04 s) 2023-08-25T04:45:39.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:39 smithi162 ceph-mon[130941]: pgmap v20: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 74 MiB/s wr, 48 op/s 2023-08-25T04:45:39.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:39 smithi183 ceph-mon[131414]: pgmap v20: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 74 MiB/s wr, 48 op/s 2023-08-25T04:45:40.062 INFO:teuthology.orchestra.run.smithi145.stderr:7100000 of 50000000 tuples (14%) done (elapsed 17.53 s, remaining 105.93 s) 2023-08-25T04:45:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:39 smithi145 ceph-mon[97681]: pgmap v20: 129 pgs: 129 active+clean; 620 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 74 MiB/s wr, 48 op/s 2023-08-25T04:45:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:7200000 of 50000000 tuples (14%) done (elapsed 17.83 s, remaining 106.00 s) 2023-08-25T04:45:40.530 INFO:teuthology.orchestra.run.smithi145.stderr:7300000 of 50000000 tuples (14%) done (elapsed 18.00 s, remaining 105.28 s) 2023-08-25T04:45:40.801 INFO:teuthology.orchestra.run.smithi145.stderr:7400000 of 50000000 tuples (14%) done (elapsed 18.27 s, remaining 105.18 s) 2023-08-25T04:45:41.097 INFO:teuthology.orchestra.run.smithi145.stderr:7500000 of 50000000 tuples (15%) done (elapsed 18.57 s, remaining 105.21 s) 2023-08-25T04:45:41.197 INFO:teuthology.orchestra.run.smithi145.stderr:7600000 of 50000000 tuples (15%) done (elapsed 18.67 s, remaining 104.14 s) 2023-08-25T04:45:41.476 INFO:teuthology.orchestra.run.smithi145.stderr:7700000 of 50000000 tuples (15%) done (elapsed 18.95 s, remaining 104.08 s) 2023-08-25T04:45:41.834 INFO:teuthology.orchestra.run.smithi145.stderr:7800000 of 50000000 tuples (15%) done (elapsed 19.30 s, remaining 104.44 s) 2023-08-25T04:45:41.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:41 smithi162 ceph-mon[130941]: pgmap v21: 129 pgs: 129 active+clean; 1.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 132 MiB/s wr, 76 op/s 2023-08-25T04:45:41.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:41 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:41.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:41 smithi183 ceph-mon[131414]: pgmap v21: 129 pgs: 129 active+clean; 1.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 132 MiB/s wr, 76 op/s 2023-08-25T04:45:41.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:41 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:41.940 INFO:teuthology.orchestra.run.smithi145.stderr:7900000 of 50000000 tuples (15%) done (elapsed 19.41 s, remaining 103.44 s) 2023-08-25T04:45:42.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:41 smithi145 ceph-mon[97681]: pgmap v21: 129 pgs: 129 active+clean; 1.2 GiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 132 MiB/s wr, 76 op/s 2023-08-25T04:45:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:41 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:42.241 INFO:teuthology.orchestra.run.smithi145.stderr:8000000 of 50000000 tuples (16%) done (elapsed 19.71 s, remaining 103.48 s) 2023-08-25T04:45:42.420 INFO:teuthology.orchestra.run.smithi145.stderr:8100000 of 50000000 tuples (16%) done (elapsed 19.89 s, remaining 102.89 s) 2023-08-25T04:45:42.688 INFO:teuthology.orchestra.run.smithi145.stderr:8200000 of 50000000 tuples (16%) done (elapsed 20.16 s, remaining 102.75 s) 2023-08-25T04:45:42.973 INFO:teuthology.orchestra.run.smithi145.stderr:8300000 of 50000000 tuples (16%) done (elapsed 20.44 s, remaining 102.71 s) 2023-08-25T04:45:43.156 INFO:teuthology.orchestra.run.smithi145.stderr:8400000 of 50000000 tuples (16%) done (elapsed 20.63 s, remaining 102.15 s) 2023-08-25T04:45:43.417 INFO:teuthology.orchestra.run.smithi145.stderr:8500000 of 50000000 tuples (17%) done (elapsed 20.89 s, remaining 101.98 s) 2023-08-25T04:45:43.707 INFO:teuthology.orchestra.run.smithi145.stderr:8600000 of 50000000 tuples (17%) done (elapsed 21.18 s, remaining 101.94 s) 2023-08-25T04:45:43.878 INFO:teuthology.orchestra.run.smithi145.stderr:8700000 of 50000000 tuples (17%) done (elapsed 21.35 s, remaining 101.34 s) 2023-08-25T04:45:43.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:43 smithi162 ceph-mon[130941]: pgmap v22: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 144 MiB/s wr, 79 op/s 2023-08-25T04:45:43.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:43 smithi183 ceph-mon[131414]: pgmap v22: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 144 MiB/s wr, 79 op/s 2023-08-25T04:45:44.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:43 smithi145 ceph-mon[97681]: pgmap v22: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 144 MiB/s wr, 79 op/s 2023-08-25T04:45:44.155 INFO:teuthology.orchestra.run.smithi145.stderr:8800000 of 50000000 tuples (17%) done (elapsed 21.62 s, remaining 101.24 s) 2023-08-25T04:45:44.423 INFO:teuthology.orchestra.run.smithi145.stderr:8900000 of 50000000 tuples (17%) done (elapsed 21.89 s, remaining 101.10 s) 2023-08-25T04:45:44.589 INFO:teuthology.orchestra.run.smithi145.stderr:9000000 of 50000000 tuples (18%) done (elapsed 22.06 s, remaining 100.49 s) 2023-08-25T04:45:44.862 INFO:teuthology.orchestra.run.smithi145.stderr:9100000 of 50000000 tuples (18%) done (elapsed 22.33 s, remaining 100.37 s) 2023-08-25T04:45:44.977 INFO:teuthology.orchestra.run.smithi145.stderr:9200000 of 50000000 tuples (18%) done (elapsed 22.45 s, remaining 99.54 s) 2023-08-25T04:45:45.281 INFO:teuthology.orchestra.run.smithi145.stderr:9300000 of 50000000 tuples (18%) done (elapsed 22.75 s, remaining 99.55 s) 2023-08-25T04:45:45.551 INFO:teuthology.orchestra.run.smithi145.stderr:9400000 of 50000000 tuples (18%) done (elapsed 23.02 s, remaining 99.43 s) 2023-08-25T04:45:45.706 INFO:teuthology.orchestra.run.smithi145.stderr:9500000 of 50000000 tuples (19%) done (elapsed 23.18 s, remaining 98.80 s) 2023-08-25T04:45:45.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:45 smithi183 ceph-mon[131414]: pgmap v23: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 127 MiB/s wr, 62 op/s 2023-08-25T04:45:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:9600000 of 50000000 tuples (19%) done (elapsed 23.46 s, remaining 98.74 s) 2023-08-25T04:45:46.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:45 smithi145 ceph-mon[97681]: pgmap v23: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 127 MiB/s wr, 62 op/s 2023-08-25T04:45:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:45 smithi162 ceph-mon[130941]: pgmap v23: 129 pgs: 129 active+clean; 1.5 GiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 127 MiB/s wr, 62 op/s 2023-08-25T04:45:46.266 INFO:teuthology.orchestra.run.smithi145.stderr:9700000 of 50000000 tuples (19%) done (elapsed 23.74 s, remaining 98.62 s) 2023-08-25T04:45:46.499 INFO:teuthology.orchestra.run.smithi145.stderr:9800000 of 50000000 tuples (19%) done (elapsed 23.97 s, remaining 98.32 s) 2023-08-25T04:45:47.031 INFO:teuthology.orchestra.run.smithi145.stderr:9900000 of 50000000 tuples (19%) done (elapsed 24.50 s, remaining 99.24 s) 2023-08-25T04:45:47.738 INFO:teuthology.orchestra.run.smithi145.stderr:10000000 of 50000000 tuples (20%) done (elapsed 25.21 s, remaining 100.83 s) 2023-08-25T04:45:47.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:47 smithi183 ceph-mon[131414]: pgmap v24: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 163 MiB/s wr, 91 op/s 2023-08-25T04:45:47.969 INFO:teuthology.orchestra.run.smithi145.stderr:10100000 of 50000000 tuples (20%) done (elapsed 25.44 s, remaining 100.50 s) 2023-08-25T04:45:48.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:47 smithi145 ceph-mon[97681]: pgmap v24: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 163 MiB/s wr, 91 op/s 2023-08-25T04:45:48.125 INFO:teuthology.orchestra.run.smithi145.stderr:10200000 of 50000000 tuples (20%) done (elapsed 25.59 s, remaining 99.87 s) 2023-08-25T04:45:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:47 smithi162 ceph-mon[130941]: pgmap v24: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 163 MiB/s wr, 91 op/s 2023-08-25T04:45:48.364 INFO:teuthology.orchestra.run.smithi145.stderr:10300000 of 50000000 tuples (20%) done (elapsed 25.83 s, remaining 99.57 s) 2023-08-25T04:45:48.497 INFO:teuthology.orchestra.run.smithi145.stderr:10400000 of 50000000 tuples (20%) done (elapsed 25.97 s, remaining 98.87 s) 2023-08-25T04:45:48.675 INFO:teuthology.orchestra.run.smithi145.stderr:10500000 of 50000000 tuples (21%) done (elapsed 26.14 s, remaining 98.35 s) 2023-08-25T04:45:48.854 INFO:teuthology.orchestra.run.smithi145.stderr:10600000 of 50000000 tuples (21%) done (elapsed 26.32 s, remaining 97.84 s) 2023-08-25T04:45:49.031 INFO:teuthology.orchestra.run.smithi145.stderr:10700000 of 50000000 tuples (21%) done (elapsed 26.50 s, remaining 97.34 s) 2023-08-25T04:45:49.153 INFO:teuthology.orchestra.run.smithi145.stderr:10800000 of 50000000 tuples (21%) done (elapsed 26.62 s, remaining 96.63 s) 2023-08-25T04:45:49.340 INFO:teuthology.orchestra.run.smithi145.stderr:10900000 of 50000000 tuples (21%) done (elapsed 26.81 s, remaining 96.17 s) 2023-08-25T04:45:49.472 INFO:teuthology.orchestra.run.smithi145.stderr:11000000 of 50000000 tuples (22%) done (elapsed 26.94 s, remaining 95.51 s) 2023-08-25T04:45:49.675 INFO:teuthology.orchestra.run.smithi145.stderr:11100000 of 50000000 tuples (22%) done (elapsed 27.14 s, remaining 95.13 s) 2023-08-25T04:45:49.858 INFO:teuthology.orchestra.run.smithi145.stderr:11200000 of 50000000 tuples (22%) done (elapsed 27.33 s, remaining 94.67 s) 2023-08-25T04:45:49.996 INFO:teuthology.orchestra.run.smithi145.stderr:11300000 of 50000000 tuples (22%) done (elapsed 27.47 s, remaining 94.06 s) 2023-08-25T04:45:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:49 smithi145 ceph-mon[97681]: pgmap v25: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 127 MiB/s wr, 65 op/s 2023-08-25T04:45:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:49 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:49 smithi162 ceph-mon[130941]: pgmap v25: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 127 MiB/s wr, 65 op/s 2023-08-25T04:45:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:49 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:49 smithi183 ceph-mon[131414]: pgmap v25: 129 pgs: 129 active+clean; 1.7 GiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 127 MiB/s wr, 65 op/s 2023-08-25T04:45:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:49 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:50.174 INFO:teuthology.orchestra.run.smithi145.stderr:11400000 of 50000000 tuples (22%) done (elapsed 27.64 s, remaining 93.60 s) 2023-08-25T04:45:50.346 INFO:teuthology.orchestra.run.smithi145.stderr:11500000 of 50000000 tuples (23%) done (elapsed 27.82 s, remaining 93.12 s) 2023-08-25T04:45:50.546 INFO:teuthology.orchestra.run.smithi145.stderr:11600000 of 50000000 tuples (23%) done (elapsed 28.02 s, remaining 92.74 s) 2023-08-25T04:45:50.690 INFO:teuthology.orchestra.run.smithi145.stderr:11700000 of 50000000 tuples (23%) done (elapsed 28.16 s, remaining 92.18 s) 2023-08-25T04:45:50.842 INFO:teuthology.orchestra.run.smithi145.stderr:11800000 of 50000000 tuples (23%) done (elapsed 28.31 s, remaining 91.65 s) 2023-08-25T04:45:51.039 INFO:teuthology.orchestra.run.smithi145.stderr:11900000 of 50000000 tuples (23%) done (elapsed 28.51 s, remaining 91.28 s) 2023-08-25T04:45:51.140 INFO:teuthology.orchestra.run.smithi145.stderr:12000000 of 50000000 tuples (24%) done (elapsed 28.61 s, remaining 90.60 s) 2023-08-25T04:45:51.330 INFO:teuthology.orchestra.run.smithi145.stderr:12100000 of 50000000 tuples (24%) done (elapsed 28.80 s, remaining 90.21 s) 2023-08-25T04:45:51.481 INFO:teuthology.orchestra.run.smithi145.stderr:12200000 of 50000000 tuples (24%) done (elapsed 28.95 s, remaining 89.70 s) 2023-08-25T04:45:51.705 INFO:teuthology.orchestra.run.smithi145.stderr:12300000 of 50000000 tuples (24%) done (elapsed 29.17 s, remaining 89.42 s) 2023-08-25T04:45:51.815 INFO:teuthology.orchestra.run.smithi145.stderr:12400000 of 50000000 tuples (24%) done (elapsed 29.29 s, remaining 88.80 s) 2023-08-25T04:45:52.000 INFO:teuthology.orchestra.run.smithi145.stderr:12500000 of 50000000 tuples (25%) done (elapsed 29.47 s, remaining 88.41 s) 2023-08-25T04:45:52.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:51 smithi145 ceph-mon[97681]: pgmap v26: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 194 MiB/s wr, 100 op/s 2023-08-25T04:45:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:51 smithi162 ceph-mon[130941]: pgmap v26: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 194 MiB/s wr, 100 op/s 2023-08-25T04:45:52.170 INFO:teuthology.orchestra.run.smithi145.stderr:12600000 of 50000000 tuples (25%) done (elapsed 29.64 s, remaining 87.98 s) 2023-08-25T04:45:52.180 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:51 smithi183 ceph-mon[131414]: pgmap v26: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 194 MiB/s wr, 100 op/s 2023-08-25T04:45:52.298 INFO:teuthology.orchestra.run.smithi145.stderr:12700000 of 50000000 tuples (25%) done (elapsed 29.77 s, remaining 87.43 s) 2023-08-25T04:45:52.480 INFO:teuthology.orchestra.run.smithi145.stderr:12800000 of 50000000 tuples (25%) done (elapsed 29.95 s, remaining 87.04 s) 2023-08-25T04:45:52.648 INFO:teuthology.orchestra.run.smithi145.stderr:12900000 of 50000000 tuples (25%) done (elapsed 30.12 s, remaining 86.62 s) 2023-08-25T04:45:52.783 INFO:teuthology.orchestra.run.smithi145.stderr:13000000 of 50000000 tuples (26%) done (elapsed 30.25 s, remaining 86.10 s) 2023-08-25T04:45:52.930 INFO:teuthology.orchestra.run.smithi145.stderr:13100000 of 50000000 tuples (26%) done (elapsed 30.40 s, remaining 85.63 s) 2023-08-25T04:45:53.113 INFO:teuthology.orchestra.run.smithi145.stderr:13200000 of 50000000 tuples (26%) done (elapsed 30.58 s, remaining 85.26 s) 2023-08-25T04:45:53.246 INFO:teuthology.orchestra.run.smithi145.stderr:13300000 of 50000000 tuples (26%) done (elapsed 30.72 s, remaining 84.76 s) 2023-08-25T04:45:53.402 INFO:teuthology.orchestra.run.smithi145.stderr:13400000 of 50000000 tuples (26%) done (elapsed 30.87 s, remaining 84.32 s) 2023-08-25T04:45:53.537 INFO:teuthology.orchestra.run.smithi145.stderr:13500000 of 50000000 tuples (27%) done (elapsed 31.01 s, remaining 83.83 s) 2023-08-25T04:45:53.757 INFO:teuthology.orchestra.run.smithi145.stderr:13600000 of 50000000 tuples (27%) done (elapsed 31.23 s, remaining 83.58 s) 2023-08-25T04:45:53.919 INFO:teuthology.orchestra.run.smithi145.stderr:13700000 of 50000000 tuples (27%) done (elapsed 31.39 s, remaining 83.17 s) 2023-08-25T04:45:54.089 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:53 smithi145 ceph-mon[97681]: pgmap v27: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 135 MiB/s wr, 73 op/s 2023-08-25T04:45:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:53 smithi162 ceph-mon[130941]: pgmap v27: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 135 MiB/s wr, 73 op/s 2023-08-25T04:45:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:53 smithi183 ceph-mon[131414]: pgmap v27: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 135 MiB/s wr, 73 op/s 2023-08-25T04:45:54.410 INFO:teuthology.orchestra.run.smithi145.stderr:13800000 of 50000000 tuples (27%) done (elapsed 31.88 s, remaining 83.62 s) 2023-08-25T04:45:55.083 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:54 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:55.108 INFO:teuthology.orchestra.run.smithi145.stderr:13900000 of 50000000 tuples (27%) done (elapsed 32.58 s, remaining 84.61 s) 2023-08-25T04:45:55.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:54 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:55.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:54 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:45:55.295 INFO:teuthology.orchestra.run.smithi145.stderr:14000000 of 50000000 tuples (28%) done (elapsed 32.76 s, remaining 84.25 s) 2023-08-25T04:45:55.697 INFO:teuthology.orchestra.run.smithi145.stderr:14100000 of 50000000 tuples (28%) done (elapsed 33.17 s, remaining 84.44 s) 2023-08-25T04:45:55.893 INFO:teuthology.orchestra.run.smithi145.stderr:14200000 of 50000000 tuples (28%) done (elapsed 33.36 s, remaining 84.11 s) 2023-08-25T04:45:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:55 smithi145 ceph-mon[97681]: pgmap v28: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 MiB/s wr, 63 op/s 2023-08-25T04:45:56.112 INFO:teuthology.orchestra.run.smithi145.stderr:14300000 of 50000000 tuples (28%) done (elapsed 33.58 s, remaining 83.84 s) 2023-08-25T04:45:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:55 smithi162 ceph-mon[130941]: pgmap v28: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 MiB/s wr, 63 op/s 2023-08-25T04:45:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:55 smithi183 ceph-mon[131414]: pgmap v28: 129 pgs: 129 active+clean; 2.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 MiB/s wr, 63 op/s 2023-08-25T04:45:56.471 INFO:teuthology.orchestra.run.smithi145.stderr:14400000 of 50000000 tuples (28%) done (elapsed 33.94 s, remaining 83.91 s) 2023-08-25T04:45:57.007 INFO:teuthology.orchestra.run.smithi145.stderr:14500000 of 50000000 tuples (29%) done (elapsed 34.48 s, remaining 84.41 s) 2023-08-25T04:45:57.192 INFO:teuthology.orchestra.run.smithi145.stderr:14600000 of 50000000 tuples (29%) done (elapsed 34.66 s, remaining 84.04 s) 2023-08-25T04:45:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:14700000 of 50000000 tuples (29%) done (elapsed 34.79 s, remaining 83.55 s) 2023-08-25T04:45:57.475 INFO:teuthology.orchestra.run.smithi145.stderr:14800000 of 50000000 tuples (29%) done (elapsed 34.94 s, remaining 83.11 s) 2023-08-25T04:45:57.901 INFO:teuthology.orchestra.run.smithi145.stderr:14900000 of 50000000 tuples (29%) done (elapsed 35.37 s, remaining 83.32 s) 2023-08-25T04:45:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:57 smithi145 ceph-mon[97681]: pgmap v29: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 MiB/s wr, 87 op/s 2023-08-25T04:45:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:57 smithi162 ceph-mon[130941]: pgmap v29: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 MiB/s wr, 87 op/s 2023-08-25T04:45:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:57 smithi183 ceph-mon[131414]: pgmap v29: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 MiB/s wr, 87 op/s 2023-08-25T04:45:58.561 INFO:teuthology.orchestra.run.smithi145.stderr:15000000 of 50000000 tuples (30%) done (elapsed 36.03 s, remaining 84.07 s) 2023-08-25T04:45:58.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:45:59.052 INFO:teuthology.orchestra.run.smithi145.stderr:15100000 of 50000000 tuples (30%) done (elapsed 36.52 s, remaining 84.41 s) 2023-08-25T04:45:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 -- 172.21.15.145:0/2058368252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c072cc0 msgr2=0x7f8a5c0730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2058368252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c0730a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009990 tx=0x7f8a4c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 -- 172.21.15.145:0/2058368252 shutdown_connections 2023-08-25T04:45:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2058368252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c111880 0x7f8a5c113c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2058368252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c0730a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2058368252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c0735e0 0x7f8a5c111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 -- 172.21.15.145:0/2058368252 >> 172.21.15.145:0/2058368252 conn(0x7f8a5c0fc3a0 msgr2=0x7f8a5c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.254+0000 7f8a60fff700 1 -- 172.21.15.145:0/2058368252 shutdown_connections 2023-08-25T04:45:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.256+0000 7f8a60fff700 1 -- 172.21.15.145:0/2058368252 wait complete. 2023-08-25T04:45:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.257+0000 7f8a60fff700 1 Processor -- start 2023-08-25T04:45:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.257+0000 7f8a60fff700 1 -- start start 2023-08-25T04:45:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c0735e0 0x7f8a5c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c1a1fd0 con 0x7f8a5c111880 2023-08-25T04:45:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c1a2140 con 0x7f8a5c072cc0 2023-08-25T04:45:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a60fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c1a2300 con 0x7f8a5c0735e0 2023-08-25T04:45:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43428/0 (socket says 172.21.15.145:43428) 2023-08-25T04:45:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2059973896 learned_addr learned my addr 172.21.15.145:0/2059973896 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.258+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c0735e0 0x7f8a5c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 -- 172.21.15.145:0/2059973896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c0735e0 msgr2=0x7f8a5c19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c0735e0 0x7f8a5c19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 -- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 msgr2=0x7f8a5c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 -- 172.21.15.145:0/2059973896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0095f0 con 0x7f8a5c072cc0 2023-08-25T04:45:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c19cff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0059d0 tx=0x7f8a4c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a4b7fe700 1 -- 172.21.15.145:0/2059973896 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01d070 con 0x7f8a5c072cc0 2023-08-25T04:45:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a4b7fe700 1 -- 172.21.15.145:0/2059973896 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a4c022470 con 0x7f8a5c072cc0 2023-08-25T04:45:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a4b7fe700 1 -- 172.21.15.145:0/2059973896 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c017650 con 0x7f8a5c072cc0 2023-08-25T04:45:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c1a2580 con 0x7f8a5c072cc0 2023-08-25T04:45:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.259+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c1a27d0 con 0x7f8a5c072cc0 2023-08-25T04:45:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.263+0000 7f8a4b7fe700 1 -- 172.21.15.145:0/2059973896 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f8a4c022890 con 0x7f8a5c072cc0 2023-08-25T04:45:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.264+0000 7f8a74846b80 1 Processor -- start 2023-08-25T04:45:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.264+0000 7f8a74846b80 1 -- start start 2023-08-25T04:45:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.264+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.264+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577aca0 0x55aba5774700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.264+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba57750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a74846b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba56dc7b0 con 0x55aba5774c40 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a74846b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba56cd780 con 0x55aba577a380 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a74846b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba56cb0a0 con 0x55aba577aca0 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577aca0 0x55aba5774700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a59d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba57750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57562/0 (socket says 172.21.15.145:57562) 2023-08-25T04:45:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2852798709 learned_addr learned my addr 172.21.15.145:0/2852798709 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2852798709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577aca0 msgr2=0x55aba5774700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577aca0 0x55aba5774700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2852798709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 msgr2=0x55aba57750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba57750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2852798709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba5546630 con 0x55aba577a380 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.265+0000 7f8a4b7fe700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 0x7f8a440a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a4b7fe700 1 -- 172.21.15.145:0/2059973896 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f8a4c0ccca0 con 0x7f8a5c072cc0 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a54007530 tx=0x7f8a54009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfb8656cc73b881 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 0x7f8a440a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a48ff9700 1 -- 172.21.15.145:0/2852798709 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54009100 con 0x55aba577a380 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a48ff9700 1 -- 172.21.15.145:0/2852798709 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5400f040 con 0x55aba577a380 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.266+0000 7f8a48ff9700 1 -- 172.21.15.145:0/2852798709 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54015dd0 con 0x55aba577a380 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a5a59c700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 0x7f8a440a2690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0ca050 tx=0x7f8a4c0b9040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 -- 172.21.15.145:0/2852798709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 msgr2=0x55aba577a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a54007530 tx=0x7f8a54009920 comp rx=0 tx=0).stop 2023-08-25T04:45:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 -- 172.21.15.145:0/2852798709 shutdown_connections 2023-08-25T04:45:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba57750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577a380 0x55aba577a760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2852798709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577aca0 0x55aba5774700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 -- 172.21.15.145:0/2852798709 >> 172.21.15.145:0/2852798709 conn(0x55aba56e90e0 msgr2=0x55aba576f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 -- 172.21.15.145:0/2852798709 shutdown_connections 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.267+0000 7f8a74846b80 1 -- 172.21.15.145:0/2852798709 wait complete. 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.268+0000 7f8a74846b80 1 -- start start 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.268+0000 7f8a74846b80 1 Processor -- start 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 0x55aba577df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba577e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 0x55aba577e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba56c6990 con 0x55aba577aca0 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba5664bb0 con 0x55aba5774c40 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba56648b0 con 0x55aba577a380 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a74846b80 10 client.0 _ll_register_callbacks cb 0x55aba55aea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 0x55aba577df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 0x55aba577df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57578/0 (socket says 172.21.15.145:57578) 2023-08-25T04:45:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/2443595680 learned_addr learned my addr 172.21.15.145:0/2443595680 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba577e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.269+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 0x55aba577e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 -- 172.21.15.145:0/2443595680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 msgr2=0x55aba577df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 0x55aba577df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 -- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 msgr2=0x55aba577e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 0x55aba577e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 -- 172.21.15.145:0/2443595680 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a54009260 con 0x55aba577a380 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 0x55aba577e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba577e4a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a50011730 tx=0x7f8a50011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58b9472c7533e30 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:45:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50016070 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aba57441c0 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.270+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f8a50004830 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a50004a30 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 10 client.25696 fetch_fsmap learned FSMap version 27 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aba54f2de0 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aba57441c0 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8a5001a840 con 0x55aba577a380 2023-08-25T04:45:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 10 client.25696 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 10 client.25696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 20 client.25696 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba578e8e0 0x55aba5790d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 4 client.25696 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.271+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aba566cba0 con 0x55aba578e8e0 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.272+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba578e8e0 0x55aba5790d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.272+0000 7f8a41ffb700 10 client.25696 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.272+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba578e8e0 0x55aba5790d00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cadbf1f230662686 server_cookie=be7768c6a924ef16 in_seq=0 out_seq=0 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a41ffb700 1 -- 172.21.15.145:0/2443595680 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aba566cba0 con 0x55aba578e8e0 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a41ffb700 10 client.25696 handle_command_reply: tid=0 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 client.25696 shutdown 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 msgr2=0x55aba577e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba577e4a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a50011730 tx=0x7f8a50011a40 comp rx=0 tx=0).stop 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 shutdown_connections 2023-08-25T04:45:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577aca0 0x55aba577e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba5774c40 0x55aba577df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba578e8e0 0x55aba5790d00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 --2- 172.21.15.145:0/2443595680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba577e4a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 >> 172.21.15.145:0/2443595680 conn(0x55aba56e90e0 msgr2=0x55aba5769b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a41ffb700 0 client.25696 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 shutdown_connections 2023-08-25T04:45:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.273+0000 7f8a74846b80 1 -- 172.21.15.145:0/2443595680 wait complete. 2023-08-25T04:45:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.274+0000 7f8a74846b80 20 client.25696 trim_cache size 0 max 16384 2023-08-25T04:45:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 Processor -- start 2023-08-25T04:45:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 -- start start 2023-08-25T04:45:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba5785790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba577d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba5548920 con 0x55aba5774c40 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba57705d0 con 0x55aba577aca0 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.291+0000 7f8a74846b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba577f540 con 0x55aba577a380 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43452/0 (socket says 172.21.15.145:43452) 2023-08-25T04:45:59.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a5ad9d700 1 -- 172.21.15.145:0/1646500375 learned_addr learned my addr 172.21.15.145:0/1646500375 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba5785790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba577d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1646500375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 msgr2=0x55aba577d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba577d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1646500375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 msgr2=0x55aba5774700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1646500375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba54dbc60 con 0x55aba577a380 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba5785790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a50011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8d01183c54ab3b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.292+0000 7f8a497fa700 1 -- 172.21.15.145:0/1646500375 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50016070 con 0x55aba577a380 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a497fa700 1 -- 172.21.15.145:0/1646500375 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5001a8b0 con 0x55aba577a380 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a497fa700 1 -- 172.21.15.145:0/1646500375 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500038e0 con 0x55aba577a380 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 -- 172.21.15.145:0/1646500375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 msgr2=0x55aba5785790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba5785790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a50011d90 comp rx=0 tx=0).stop 2023-08-25T04:45:59.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 -- 172.21.15.145:0/1646500375 shutdown_connections 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba577d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba5774c40 0x55aba5774700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1646500375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba577a380 0x55aba5785790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 -- 172.21.15.145:0/1646500375 >> 172.21.15.145:0/1646500375 conn(0x55aba56e90e0 msgr2=0x55aba578ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 -- 172.21.15.145:0/1646500375 shutdown_connections 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.293+0000 7f8a74846b80 1 -- 172.21.15.145:0/1646500375 wait complete. 2023-08-25T04:45:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 -- start start 2023-08-25T04:45:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 Processor -- start 2023-08-25T04:45:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba5774c40 0x55aba5781770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba57e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba57825c0 con 0x55aba577a380 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba57e1c00 con 0x55aba577aca0 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.294+0000 7f8a74846b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba57e1d40 con 0x55aba5774c40 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a74846b80 10 client.0 _ll_register_callbacks cb 0x7f8a50009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a5ad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba5774c40 0x55aba5781770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43464/0 (socket says 172.21.15.145:43464) 2023-08-25T04:45:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1123668337 learned_addr learned my addr 172.21.15.145:0/1123668337 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba57e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1123668337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba5774c40 msgr2=0x55aba5781770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba5774c40 0x55aba5781770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1123668337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 msgr2=0x55aba57e16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba57e16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 -- 172.21.15.145:0/1123668337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aba5546630 con 0x55aba577a380 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a59d9b700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a50011700 tx=0x7f8a50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4b09a7b3a43ed6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.295+0000 7f8a41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50016040 con 0x55aba577a380 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aba57678b0 con 0x55aba577a380 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f8a50005110 con 0x55aba577a380 2023-08-25T04:45:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a500052d0 con 0x55aba577a380 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 10 client.15687 fetch_fsmap learned FSMap version 27 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aba54f2de0 con 0x55aba577a380 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aba57678b0 con 0x55aba577a380 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8a50031070 con 0x55aba577a380 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:45:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 20 client.15687 populate_metadata read hostname 'smithi145' 2023-08-25T04:45:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba57e31b0 0x55aba57e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 4 client.15687 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:45:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.296+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aba56bf970 con 0x55aba57e31b0 2023-08-25T04:45:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba57e31b0 0x55aba57e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a41ffb700 10 client.15687 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a5ad9d700 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba57e31b0 0x55aba57e55d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4854ebbc5dbd3e server_cookie=34e0be874a772335 in_seq=0 out_seq=0 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a41ffb700 1 -- 172.21.15.145:0/1123668337 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aba56bf970 con 0x55aba57e31b0 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a41ffb700 10 client.15687 handle_command_reply: tid=0 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.297+0000 7f8a74846b80 1 client.15687 shutdown 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 msgr2=0x55aba5781cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a50011700 tx=0x7f8a50005880 comp rx=0 tx=0).stop 2023-08-25T04:45:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 shutdown_connections 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55aba577aca0 0x55aba57e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55aba57e31b0 0x55aba57e55d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aba5774c40 0x55aba5781770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 --2- 172.21.15.145:0/1123668337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aba577a380 0x55aba5781cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 >> 172.21.15.145:0/1123668337 conn(0x55aba56e90e0 msgr2=0x55aba577e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a41ffb700 0 client.15687 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:45:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 shutdown_connections 2023-08-25T04:45:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 1 -- 172.21.15.145:0/1123668337 wait complete. 2023-08-25T04:45:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a74846b80 20 client.15687 trim_cache size 0 max 16384 2023-08-25T04:45:59.319 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:45:59.319 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "no active scrubs running", 2023-08-25T04:45:59.319 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": {} 2023-08-25T04:45:59.320 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 msgr2=0x7f8a440a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.298+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 0x7f8a440a2690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0ca050 tx=0x7f8a4c0b9040 comp rx=0 tx=0).stop 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 msgr2=0x7f8a5c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c19cff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0059d0 tx=0x7f8a4c005720 comp rx=0 tx=0).stop 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 shutdown_connections 2023-08-25T04:45:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5c111880 0x7f8a5c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f8a440a01d0 0x7f8a440a2690 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a5c072cc0 0x7f8a5c19cff0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 --2- 172.21.15.145:0/2059973896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a5c0735e0 0x7f8a5c19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 >> 172.21.15.145:0/2059973896 conn(0x7f8a5c0fc3a0 msgr2=0x7f8a5c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 shutdown_connections 2023-08-25T04:45:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.299+0000 7f8a60fff700 1 -- 172.21.15.145:0/2059973896 wait complete. 2023-08-25T04:45:59.322 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-08-25T04:45:59.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T04:45:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:15200000 of 50000000 tuples (30%) done (elapsed 36.80 s, remaining 84.26 s) 2023-08-25T04:45:59.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.569+0000 7feb9dfff700 1 -- 172.21.15.145:0/4141657857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 msgr2=0x7feb98106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 --2- 172.21.15.145:0/4141657857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb98106cb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb88009990 tx=0x7feb88009ca0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 -- 172.21.15.145:0/4141657857 shutdown_connections 2023-08-25T04:45:59.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 --2- 172.21.15.145:0/4141657857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb98101280 0x7feb9810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 --2- 172.21.15.145:0/4141657857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb98106cb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 --2- 172.21.15.145:0/4141657857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb981008c0 0x7feb98100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 -- 172.21.15.145:0/4141657857 >> 172.21.15.145:0/4141657857 conn(0x7feb980fc3d0 msgr2=0x7feb980fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 -- 172.21.15.145:0/4141657857 shutdown_connections 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.570+0000 7feb9dfff700 1 -- 172.21.15.145:0/4141657857 wait complete. 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.571+0000 7feb9dfff700 1 Processor -- start 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.571+0000 7feb9dfff700 1 -- start start 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb98101280 0x7feb981a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb9810c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9810cbf0 con 0x7feb981008c0 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb977fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb977fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43492/0 (socket says 172.21.15.145:43492) 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb977fe700 1 -- 172.21.15.145:0/891706679 learned_addr learned my addr 172.21.15.145:0/891706679 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9810cd60 con 0x7feb98101280 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.572+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9810cf20 con 0x7feb981068d0 2023-08-25T04:45:59.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb96ffd700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb98101280 0x7feb981a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb97fff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb9810c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 -- 172.21.15.145:0/891706679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 msgr2=0x7feb9810c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb9810c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 -- 172.21.15.145:0/891706679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb98101280 msgr2=0x7feb981a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb98101280 0x7feb981a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880095f0 con 0x7feb981008c0 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.573+0000 7feb977fe700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feb880095c0 tx=0x7feb8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.574+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8801d070 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.574+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb88005050 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.574+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88004a90 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.575+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9810d1a0 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.575+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb98196030 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.576+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb98067710 con 0x7feb981008c0 2023-08-25T04:45:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.578+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7feb88005430 con 0x7feb981008c0 2023-08-25T04:45:59.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.580+0000 7feb94ff9700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 0x7feb800a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:45:59.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.580+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7feb880cccf0 con 0x7feb981008c0 2023-08-25T04:45:59.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.581+0000 7feb96ffd700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 0x7feb800a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:45:59.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.582+0000 7feb96ffd700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 0x7feb800a26d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb9810dc30 tx=0x7feb8c0073d0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:45:59.584 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.583+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7feb88096320 con 0x7feb981008c0 2023-08-25T04:45:59.644 INFO:teuthology.orchestra.run.smithi145.stderr:15300000 of 50000000 tuples (30%) done (elapsed 37.11 s, remaining 84.17 s) 2023-08-25T04:45:59.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.936+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb9810d970 con 0x7feb981008c0 2023-08-25T04:45:59.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.938+0000 7feb94ff9700 1 -- 172.21.15.145:0/891706679 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11306 (secure 0 0 0) 0x7feb88099940 con 0x7feb981008c0 2023-08-25T04:45:59.939 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T04:45:59.939 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:45:53.678042+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T04:45:59.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 msgr2=0x7feb800a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 0x7feb800a26d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb9810dc30 tx=0x7feb8c0073d0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 msgr2=0x7feb981a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:45:59.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feb880095c0 tx=0x7feb8800f690 comp rx=0 tx=0).stop 2023-08-25T04:45:59.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 shutdown_connections 2023-08-25T04:45:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb981068d0 0x7feb9810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb981008c0 0x7feb981a1c60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7feb800a0210 0x7feb800a26d0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb9810dc30 tx=0x7feb8c0073d0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 --2- 172.21.15.145:0/891706679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb98101280 0x7feb981a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:45:59.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 >> 172.21.15.145:0/891706679 conn(0x7feb980fc3d0 msgr2=0x7feb980fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:45:59.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 shutdown_connections 2023-08-25T04:45:59.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:45:59.941+0000 7feb9dfff700 1 -- 172.21.15.145:0/891706679 wait complete. 2023-08-25T04:45:59.945 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 27 2023-08-25T04:45:59.964 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:45:59 smithi145 ceph-mon[97681]: pgmap v30: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 130 MiB/s wr, 59 op/s 2023-08-25T04:45:59.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-08-25T04:46:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:45:59 smithi162 ceph-mon[130941]: pgmap v30: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 130 MiB/s wr, 59 op/s 2023-08-25T04:46:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:45:59 smithi183 ceph-mon[131414]: pgmap v30: 129 pgs: 129 active+clean; 2.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 130 MiB/s wr, 59 op/s 2023-08-25T04:46:00.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 -- 172.21.15.145:0/3300561265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93381068b0 msgr2=0x7f9338106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 --2- 172.21.15.145:0/3300561265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93381068b0 0x7f9338106c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9328009990 tx=0x7f9328009ca0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 -- 172.21.15.145:0/3300561265 shutdown_connections 2023-08-25T04:46:00.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 --2- 172.21.15.145:0/3300561265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9338101260 0x7f933810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 --2- 172.21.15.145:0/3300561265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93381068b0 0x7f9338106c90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 --2- 172.21.15.145:0/3300561265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f9338100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.282+0000 7f933ddff700 1 -- 172.21.15.145:0/3300561265 >> 172.21.15.145:0/3300561265 conn(0x7f93380fc390 msgr2=0x7f93380fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.283+0000 7f933ddff700 1 -- 172.21.15.145:0/3300561265 shutdown_connections 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.283+0000 7f933ddff700 1 -- 172.21.15.145:0/3300561265 wait complete. 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.284+0000 7f933ddff700 1 Processor -- start 2023-08-25T04:46:00.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.284+0000 7f933ddff700 1 -- start start 2023-08-25T04:46:00.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.284+0000 7f933ddff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f933806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.284+0000 7f933ddff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9338101260 0x7f933806c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f933ddff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f933ddff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338070e30 con 0x7f93381008a0 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f933ddff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338070fa0 con 0x7f93381068b0 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f933ddff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338071160 con 0x7f9338101260 2023-08-25T04:46:00.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f93377fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f933806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57648/0 (socket says 172.21.15.145:57648) 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 -- 172.21.15.145:0/2583905931 learned_addr learned my addr 172.21.15.145:0/2583905931 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f93377fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f933806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43520/0 (socket says 172.21.15.145:43520) 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9336ffd700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9338101260 0x7f933806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 -- 172.21.15.145:0/2583905931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9338101260 msgr2=0x7f933806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9338101260 0x7f933806c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 -- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 msgr2=0x7f933806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f933806bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.285+0000 7f9337fff700 1 -- 172.21.15.145:0/2583905931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280095f0 con 0x7f93381068b0 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.286+0000 7f9337fff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9330007d70 tx=0x7f933000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.286+0000 7f9334ff9700 1 -- 172.21.15.145:0/2583905931 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933000ac90 con 0x7f93381068b0 2023-08-25T04:46:00.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.286+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9338071440 con 0x7f93381068b0 2023-08-25T04:46:00.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.286+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93381ac000 con 0x7f93381068b0 2023-08-25T04:46:00.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.289+0000 7f9334ff9700 1 -- 172.21.15.145:0/2583905931 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933000adf0 con 0x7f93381068b0 2023-08-25T04:46:00.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.289+0000 7f9334ff9700 1 -- 172.21.15.145:0/2583905931 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300115e0 con 0x7f93381068b0 2023-08-25T04:46:00.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.292+0000 7f9334ff9700 1 -- 172.21.15.145:0/2583905931 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f93300117e0 con 0x7f93381068b0 2023-08-25T04:46:00.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 Processor -- start 2023-08-25T04:46:00.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 -- start start 2023-08-25T04:46:00.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f9334ff9700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 0x7f93200a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.296 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a462644b90 0x55a462644f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f9334ff9700 1 -- 172.21.15.145:0/2583905931 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f93300cb9b0 con 0x7f93381068b0 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a46263ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46259e7b0 con 0x55a462644b90 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46258f780 con 0x55a46263f450 2023-08-25T04:46:00.297 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.294+0000 7f93516aeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46258d0a0 con 0x55a4626454b0 2023-08-25T04:46:00.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f9337fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f9337fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57650/0 (socket says 172.21.15.145:57650) 2023-08-25T04:46:00.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f9337fff700 1 -- 172.21.15.145:0/3221422400 learned_addr learned my addr 172.21.15.145:0/3221422400 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f93377fe700 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a462644b90 0x55a462644f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f9337fff700 1 -- 172.21.15.145:0/3221422400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 msgr2=0x55a46263ef10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.295+0000 7f9337fff700 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a46263ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f9337fff700 1 -- 172.21.15.145:0/3221422400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a462644b90 msgr2=0x55a462644f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f9337fff700 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a462644b90 0x55a462644f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f9337fff700 1 -- 172.21.15.145:0/3221422400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a462408630 con 0x55a46263f450 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f9336ffd700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 0x7f93200a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f9337fff700 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9330098cb0 tx=0x7f93300988e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0413d179524e79 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f931dffb700 1 -- 172.21.15.145:0/3221422400 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330098d20 con 0x55a46263f450 2023-08-25T04:46:00.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f931dffb700 1 -- 172.21.15.145:0/3221422400 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933009cb50 con 0x55a46263f450 2023-08-25T04:46:00.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.296+0000 7f931dffb700 1 -- 172.21.15.145:0/3221422400 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330098f80 con 0x55a46263f450 2023-08-25T04:46:00.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.297+0000 7f93516aeb80 1 -- 172.21.15.145:0/3221422400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 msgr2=0x55a46263f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.297+0000 7f9336ffd700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 0x7f93200a26a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55a462645910 tx=0x7f932c00b410 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.297+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9330098cb0 tx=0x7f93300988e0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 -- 172.21.15.145:0/3221422400 shutdown_connections 2023-08-25T04:46:00.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a46263f450 0x55a46263f8b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a462644b90 0x55a462644f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3221422400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a46263ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 -- 172.21.15.145:0/3221422400 >> 172.21.15.145:0/3221422400 conn(0x55a4625ab0e0 msgr2=0x55a462631410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 -- 172.21.15.145:0/3221422400 shutdown_connections 2023-08-25T04:46:00.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.298+0000 7f93516aeb80 1 -- 172.21.15.145:0/3221422400 wait complete. 2023-08-25T04:46:00.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.299+0000 7f93516aeb80 1 -- start start 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.299+0000 7f93516aeb80 1 Processor -- start 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a46264b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a4626454b0 0x55a462648d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a462588990 con 0x55a46263f450 2023-08-25T04:46:00.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a462526bb0 con 0x55a4626454b0 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4625268b0 con 0x55a462644b90 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93516aeb80 10 client.0 _ll_register_callbacks cb 0x55a462470a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.300+0000 7f93377fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a46264b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f93377fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a46264b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43542/0 (socket says 172.21.15.145:43542) 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f93377fe700 1 -- 172.21.15.145:0/454152518 learned_addr learned my addr 172.21.15.145:0/454152518 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41694/0 (socket says 172.21.15.145:41694) 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9337fff700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a4626454b0 0x55a462648d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 -- 172.21.15.145:0/454152518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a4626454b0 msgr2=0x55a462648d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a4626454b0 0x55a462648d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 -- 172.21.15.145:0/454152518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 msgr2=0x55a46264b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a46264b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 -- 172.21.15.145:0/454152518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93381ac000 con 0x55a462644b90 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9336ffd700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932c005bf0 tx=0x7f932c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0a0ab1b262490c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.301+0000 7f9316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c01e620 con 0x55a462644b90 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f932c017070 con 0x55a462644b90 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4626061c0 con 0x55a462644b90 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f932c01e780 con 0x55a462644b90 2023-08-25T04:46:00.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f93516aeb80 10 client.25723 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.302+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a4623b4de0 con 0x55a462644b90 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.303+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4626061c0 con 0x55a462644b90 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.304+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f932c01e8e0 con 0x55a462644b90 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.304+0000 7f93516aeb80 10 client.25723 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.304+0000 7f93516aeb80 10 client.25723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-08-25T04:46:00.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.305+0000 7f93516aeb80 20 client.25723 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:00.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.305+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626376d0 0x55a462639af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.305+0000 7f93516aeb80 4 client.25723 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:00.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.305+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a46252eba0 con 0x55a4626376d0 2023-08-25T04:46:00.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.305+0000 7f93377fe700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626376d0 0x55a462639af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f93377fe700 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626376d0 0x55a462639af0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccd7bc1a9ba151c4 server_cookie=15367bf0832c3c6e in_seq=0 out_seq=0 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f9316ffd700 10 client.25723 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f9316ffd700 1 -- 172.21.15.145:0/454152518 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a46252eba0 con 0x55a4626376d0 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f9316ffd700 10 client.25723 handle_command_reply: tid=0 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f93516aeb80 1 client.25723 shutdown 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 msgr2=0x55a462648690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.306+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932c005bf0 tx=0x7f932c0047c0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 shutdown_connections 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626376d0 0x55a462639af0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a4626454b0 0x55a462648d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a46264b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 --2- 172.21.15.145:0/454152518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a462644b90 0x55a462648690 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 >> 172.21.15.145:0/454152518 conn(0x55a4625ab0e0 msgr2=0x55a46262bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f9316ffd700 0 client.25723 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:46:00.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 shutdown_connections 2023-08-25T04:46:00.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 1 -- 172.21.15.145:0/454152518 wait complete. 2023-08-25T04:46:00.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.307+0000 7f93516aeb80 20 client.25723 trim_cache size 0 max 16384 2023-08-25T04:46:00.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.325+0000 7f93516aeb80 1 Processor -- start 2023-08-25T04:46:00.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.325+0000 7f93516aeb80 1 -- start start 2023-08-25T04:46:00.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 0x55a462647e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a462648390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 0x55a462641620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46240a920 con 0x55a4626454b0 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46262dc80 con 0x55a462644b90 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93516aeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a462646bf0 con 0x55a46263f450 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f9337fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 0x55a462641620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f9337fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 0x55a462641620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43558/0 (socket says 172.21.15.145:43558) 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f9337fff700 1 -- 172.21.15.145:0/972093630 learned_addr learned my addr 172.21.15.145:0/972093630 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f93377fe700 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 0x55a462647e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.326+0000 7f9336ffd700 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a462648390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 -- 172.21.15.145:0/972093630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 msgr2=0x55a462648390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a462648390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 -- 172.21.15.145:0/972093630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 msgr2=0x55a462641620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 0x55a462641620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 -- 172.21.15.145:0/972093630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a46239dc60 con 0x55a46263f450 2023-08-25T04:46:00.331 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:46:00.088+0000 7f0df7629700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:46:00.331 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:46:00.088+0000 7f0df7629700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:46:00.331 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:46:00.088+0000 7f0df7629700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:46:00.332 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:46:00.088+0000 7f0df7629700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:46:00.332 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T04:46:00.088+0000 7f0df7629700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-25T04:46:00.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.327+0000 7f93377fe700 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 0x55a462647e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a46263a0d0 tx=0x7f9328005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654f4ff4ded96db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.328+0000 7f931e7fc700 1 -- 172.21.15.145:0/972093630 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932801d040 con 0x55a46263f450 2023-08-25T04:46:00.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.328+0000 7f931e7fc700 1 -- 172.21.15.145:0/972093630 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932801e040 con 0x55a46263f450 2023-08-25T04:46:00.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.328+0000 7f931e7fc700 1 -- 172.21.15.145:0/972093630 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328003af0 con 0x55a46263f450 2023-08-25T04:46:00.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 -- 172.21.15.145:0/972093630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 msgr2=0x55a462647e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 0x55a462647e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a46263a0d0 tx=0x7f9328005fd0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 -- 172.21.15.145:0/972093630 shutdown_connections 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4626454b0 0x55a462641620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a46263f450 0x55a462647e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 --2- 172.21.15.145:0/972093630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a462648390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 -- 172.21.15.145:0/972093630 >> 172.21.15.145:0/972093630 conn(0x55a4625ab0e0 msgr2=0x55a4626427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 -- 172.21.15.145:0/972093630 shutdown_connections 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.329+0000 7f93516aeb80 1 -- 172.21.15.145:0/972093630 wait complete. 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.330+0000 7f93516aeb80 1 -- start start 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.330+0000 7f93516aeb80 1 Processor -- start 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a462641380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a4626418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4626b5190 con 0x55a46263f450 2023-08-25T04:46:00.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4626b52d0 con 0x55a462644b90 2023-08-25T04:46:00.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4626b5410 con 0x55a4626454b0 2023-08-25T04:46:00.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93516aeb80 10 client.0 _ll_register_callbacks cb 0x7f932c00bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a4626418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f93377fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a462641380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f9337fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f9337fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41710/0 (socket says 172.21.15.145:41710) 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.331+0000 7f9337fff700 1 -- 172.21.15.145:0/3697672228 learned_addr learned my addr 172.21.15.145:0/3697672228 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 -- 172.21.15.145:0/3697672228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 msgr2=0x55a4626418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a4626418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 -- 172.21.15.145:0/3697672228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 msgr2=0x55a462641380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a462641380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 -- 172.21.15.145:0/3697672228 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933810e070 con 0x55a4626454b0 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9337fff700 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9330098cb0 tx=0x7f93300060c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ca108fbee3e695 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300184c0 con 0x55a4626454b0 2023-08-25T04:46:00.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4626298b0 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f933009cb50 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9330098210 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f93516aeb80 10 client.25729 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a4623db250 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.332+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4626298b0 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f93300a06a0 con 0x55a4626454b0 2023-08-25T04:46:00.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:00.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 10 client.25729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 20 client.25729 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626b6780 0x55a4626b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 4 client.25729 mds_command: new command op to 24532 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 --> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a462581970 con 0x55a4626b6780 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.333+0000 7f93377fe700 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626b6780 0x55a4626b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f93377fe700 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626b6780 0x55a4626b8ba0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f97ac7a7f1ac809a server_cookie=40b2667c7e049e73 in_seq=0 out_seq=0 2023-08-25T04:46:00.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f9316ffd700 10 client.25729 ms_handle_connect on v2:172.21.15.183:6838/3401840698 2023-08-25T04:46:00.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f9316ffd700 1 -- 172.21.15.145:0/3697672228 <== mds.2 v2:172.21.15.183:6838/3401840698 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a462581970 con 0x55a4626b6780 2023-08-25T04:46:00.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f9316ffd700 10 client.25729 handle_command_reply: tid=0 2023-08-25T04:46:00.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f93516aeb80 1 client.25729 shutdown 2023-08-25T04:46:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 msgr2=0x55a462641e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.334+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9330098cb0 tx=0x7f93300060c0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 shutdown_connections 2023-08-25T04:46:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:6838/3401840698,v1:172.21.15.183:6839/3401840698] conn(0x55a4626b6780 0x55a4626b8ba0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a4626454b0 0x55a462641e00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a46263f450 0x55a462641380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f9316ffd700 0 client.25729 ms_handle_reset on v2:172.21.15.183:6838/3401840698 2023-08-25T04:46:00.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 --2- 172.21.15.145:0/3697672228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a462644b90 0x55a4626418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 >> 172.21.15.145:0/3697672228 conn(0x55a4625ab0e0 msgr2=0x55a4626461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 shutdown_connections 2023-08-25T04:46:00.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 1 -- 172.21.15.145:0/3697672228 wait complete. 2023-08-25T04:46:00.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.335+0000 7f93516aeb80 20 client.25729 trim_cache size 0 max 16384 2023-08-25T04:46:00.352 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:46:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 msgr2=0x7f93200a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 0x7f93200a26a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55a462645910 tx=0x7f932c00b410 comp rx=0 tx=0).stop 2023-08-25T04:46:00.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 msgr2=0x7f9338070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9330007d70 tx=0x7f933000a340 comp rx=0 tx=0).stop 2023-08-25T04:46:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 shutdown_connections 2023-08-25T04:46:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f93200a01e0 0x7f93200a26a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93381068b0 0x7f9338070860 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93381008a0 0x7f933806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 --2- 172.21.15.145:0/2583905931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9338101260 0x7f933806c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 >> 172.21.15.145:0/2583905931 conn(0x7f93380fc390 msgr2=0x7f93380fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 shutdown_connections 2023-08-25T04:46:00.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.336+0000 7f933ddff700 1 -- 172.21.15.145:0/2583905931 wait complete. 2023-08-25T04:46:00.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-08-25T04:46:00.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 -- 172.21.15.145:0/2243023742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de0072cc0 msgr2=0x7f9de00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/2243023742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de0072cc0 0x7f9de00730a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b0a0 tx=0x7f9dd000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 -- 172.21.15.145:0/2243023742 shutdown_connections 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/2243023742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0111720 0x7f9de0113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/2243023742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de0072cc0 0x7f9de00730a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/2243023742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de00735e0 0x7f9de01111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 -- 172.21.15.145:0/2243023742 >> 172.21.15.145:0/2243023742 conn(0x7f9de00fc1e0 msgr2=0x7f9de00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 -- 172.21.15.145:0/2243023742 shutdown_connections 2023-08-25T04:46:00.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.630+0000 7f9de4d9d700 1 -- 172.21.15.145:0/2243023742 wait complete. 2023-08-25T04:46:00.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.631+0000 7f9de4d9d700 1 Processor -- start 2023-08-25T04:46:00.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 -- start start 2023-08-25T04:46:00.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de00735e0 0x7f9de019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de0111720 0x7f9de01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01a1f00 con 0x7f9de0111720 2023-08-25T04:46:00.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01a2070 con 0x7f9de00735e0 2023-08-25T04:46:00.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.632+0000 7f9de4d9d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01a2230 con 0x7f9de0072cc0 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41726/0 (socket says 172.21.15.145:41726) 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 -- 172.21.15.145:0/3718125598 learned_addr learned my addr 172.21.15.145:0/3718125598 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de0111720 0x7f9de01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 -- 172.21.15.145:0/3718125598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de00735e0 msgr2=0x7f9de019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de00735e0 0x7f9de019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 -- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de0111720 msgr2=0x7f9de01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de0111720 0x7f9de01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 -- 172.21.15.145:0/3718125598 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009d00 con 0x7f9de0072cc0 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.633+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b070 tx=0x7f9dd000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.634+0000 7f9ddd7fa700 1 -- 172.21.15.145:0/3718125598 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000e040 con 0x7f9de0072cc0 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.634+0000 7f9ddd7fa700 1 -- 172.21.15.145:0/3718125598 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dd0007930 con 0x7f9de0072cc0 2023-08-25T04:46:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.634+0000 7f9ddd7fa700 1 -- 172.21.15.145:0/3718125598 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd001b820 con 0x7f9de0072cc0 2023-08-25T04:46:00.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.634+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01a24b0 con 0x7f9de0072cc0 2023-08-25T04:46:00.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.634+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de01a2730 con 0x7f9de0072cc0 2023-08-25T04:46:00.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.638+0000 7f9ddd7fa700 1 -- 172.21.15.145:0/3718125598 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) v1 ==== 134848+0+0 (secure 0 0 0) 0x7f9dd001da30 con 0x7f9de0072cc0 2023-08-25T04:46:00.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 Processor -- start 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 -- start start 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0500 0x560e9fdc08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 0x560e9fdba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fd227b0 con 0x560e9fdc0e20 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fd13780 con 0x560e9fdc0500 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.639+0000 7f9df9876b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fd110a0 con 0x560e9fdbac40 2023-08-25T04:46:00.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41734/0 (socket says 172.21.15.145:41734) 2023-08-25T04:46:00.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 -- 172.21.15.145:0/1414415024 learned_addr learned my addr 172.21.15.145:0/1414415024 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9ddffff700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0500 0x560e9fdc08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 -- 172.21.15.145:0/1414415024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0500 msgr2=0x560e9fdc08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0500 0x560e9fdc08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.640+0000 7f9dcffff700 1 -- 172.21.15.145:0/1414415024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 msgr2=0x560e9fdba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9ddd7fa700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 0x7f9dc80a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 0x560e9fdba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9dcffff700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 0x560e9fdba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9dcffff700 1 -- 172.21.15.145:0/1414415024 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9fb8c630 con 0x560e9fdbac40 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 0x7f9dc80a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9ddd7fa700 1 -- 172.21.15.145:0/3718125598 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9dd00cc5a0 con 0x7f9de0072cc0 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.641+0000 7f9dcffff700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bb60 tx=0x7f9dd400be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13bd56c99859df9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.642+0000 7f9dcdffb700 1 -- 172.21.15.145:0/1414415024 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400d890 con 0x560e9fdbac40 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.642+0000 7f9dcdffb700 1 -- 172.21.15.145:0/1414415024 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dd4004830 con 0x560e9fdbac40 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.642+0000 7f9dcdffb700 1 -- 172.21.15.145:0/1414415024 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4005540 con 0x560e9fdbac40 2023-08-25T04:46:00.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.642+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 0x560e9fdba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- 172.21.15.145:0/1414415024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 msgr2=0x560e9fdbb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 0x7f9dc80a2730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9de0069d30 tx=0x7f9dd8009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bb60 tx=0x7f9dd400be70 comp rx=0 tx=0).stop 2023-08-25T04:46:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- 172.21.15.145:0/1414415024 shutdown_connections 2023-08-25T04:46:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdbac40 0x560e9fdbb080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0500 0x560e9fdc08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 --2- 172.21.15.145:0/1414415024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0e20 0x560e9fdba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- 172.21.15.145:0/1414415024 >> 172.21.15.145:0/1414415024 conn(0x560e9fd2f0e0 msgr2=0x560e9fdb5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- 172.21.15.145:0/1414415024 shutdown_connections 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- 172.21.15.145:0/1414415024 wait complete. 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.643+0000 7f9df9876b80 1 -- start start 2023-08-25T04:46:00.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 Processor -- start 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdbac40 0x560e9fdc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0500 0x560e9fdc4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fd0c990 con 0x560e9fdc0500 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fcaabb0 con 0x560e9fdbac40 2023-08-25T04:46:00.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fcaa8b0 con 0x560e9fdc0e20 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9df9876b80 10 client.0 _ll_register_callbacks cb 0x560e9fbf4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41742/0 (socket says 172.21.15.145:41742) 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.644+0000 7f9ddffff700 1 -- 172.21.15.145:0/3517102345 learned_addr learned my addr 172.21.15.145:0/3517102345 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9dcffff700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0500 0x560e9fdc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdbac40 0x560e9fdc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 -- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdbac40 msgr2=0x560e9fdc6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdbac40 0x560e9fdc6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 -- 172.21.15.145:0/3517102345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0500 msgr2=0x560e9fdc4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0500 0x560e9fdc4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 -- 172.21.15.145:0/3517102345 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd400b7c0 con 0x560e9fdc0e20 2023-08-25T04:46:00.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009ee0 tx=0x7f9dd8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8334b589e625b25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9dbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:00.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd801b660 con 0x560e9fdc0e20 2023-08-25T04:46:00.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e9fd8a1c0 con 0x560e9fdc0e20 2023-08-25T04:46:00.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.645+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9dd801b7c0 con 0x560e9fdc0e20 2023-08-25T04:46:00.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.646+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dd801b980 con 0x560e9fdc0e20 2023-08-25T04:46:00.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.646+0000 7f9df9876b80 10 client.25741 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.646+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e9fb38de0 con 0x560e9fdc0e20 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.646+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dd802f080 con 0x560e9fdc0e20 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.646+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9dd8010d20 con 0x560e9fdc0e20 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 10 client.25741 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 20 client.25741 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:00.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fdd4920 0x560e9fdd6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 4 client.25741 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e9fcb2ba0 con 0x560e9fdd4920 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fdd4920 0x560e9fdd6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fdd4920 0x560e9fdd6d40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc4f20d16cae8600 server_cookie=d7a2113943acd4cc in_seq=0 out_seq=0 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.647+0000 7f9dbeffd700 10 client.25741 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.648+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3517102345 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e9fcb2ba0 con 0x560e9fdd4920 2023-08-25T04:46:00.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.648+0000 7f9dbeffd700 10 client.25741 handle_command_reply: tid=0 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.648+0000 7f9df9876b80 1 client.25741 shutdown 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.648+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 msgr2=0x560e9fdc4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.648+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009ee0 tx=0x7f9dd8004750 comp rx=0 tx=0).stop 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 shutdown_connections 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fdd4920 0x560e9fdd6d40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0e20 0x560e9fdc4680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdbac40 0x560e9fdc6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3517102345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdc0500 0x560e9fdc4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 >> 172.21.15.145:0/3517102345 conn(0x560e9fd2f0e0 msgr2=0x560e9fdb96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9dbeffd700 0 client.25741 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 shutdown_connections 2023-08-25T04:46:00.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 1 -- 172.21.15.145:0/3517102345 wait complete. 2023-08-25T04:46:00.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.649+0000 7f9df9876b80 20 client.25741 trim_cache size 0 max 16384 2023-08-25T04:46:00.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.666+0000 7f9df9876b80 1 Processor -- start 2023-08-25T04:46:00.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.666+0000 7f9df9876b80 1 -- start start 2023-08-25T04:46:00.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.666+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fdcb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.666+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fdc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9df9876b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fb8e920 con 0x560e9fdbac40 2023-08-25T04:46:00.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9df9876b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fdbb990 con 0x560e9fdc0e20 2023-08-25T04:46:00.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9df9876b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fdb1840 con 0x560e9fdc0500 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9dcffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fdc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fdcb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57730/0 (socket says 172.21.15.145:57730) 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 -- 172.21.15.145:0/2689835114 learned_addr learned my addr 172.21.15.145:0/2689835114 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 -- 172.21.15.145:0/2689835114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 msgr2=0x560e9fdc6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fdc6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 -- 172.21.15.145:0/2689835114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 msgr2=0x560e9fdcb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fdcb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.667+0000 7f9ddffff700 1 -- 172.21.15.145:0/2689835114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9fdaedf0 con 0x560e9fdc0e20 2023-08-25T04:46:00.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.668+0000 7f9ddffff700 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8014010 tx=0x7f9dd80045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81e50687bec25e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.668+0000 7f9dce7fc700 1 -- 172.21.15.145:0/2689835114 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8012ac0 con 0x560e9fdc0e20 2023-08-25T04:46:00.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.668+0000 7f9dce7fc700 1 -- 172.21.15.145:0/2689835114 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e9fd8a1c0 con 0x560e9fdc0e20 2023-08-25T04:46:00.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.668+0000 7f9dce7fc700 1 -- 172.21.15.145:0/2689835114 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8005780 con 0x560e9fdc0e20 2023-08-25T04:46:00.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.668+0000 7f9df9876b80 1 -- 172.21.15.145:0/2689835114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 msgr2=0x560e9fdbdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8014010 tx=0x7f9dd80045f0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 -- 172.21.15.145:0/2689835114 shutdown_connections 2023-08-25T04:46:00.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fdbdf70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fdcb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 --2- 172.21.15.145:0/2689835114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fdc6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 -- 172.21.15.145:0/2689835114 >> 172.21.15.145:0/2689835114 conn(0x560e9fd2f0e0 msgr2=0x560e9fdd00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 -- 172.21.15.145:0/2689835114 shutdown_connections 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 -- 172.21.15.145:0/2689835114 wait complete. 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.669+0000 7f9df9876b80 1 -- start start 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 Processor -- start 2023-08-25T04:46:00.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fe1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fe27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fe1d4f0 con 0x560e9fdbac40 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fe27ca0 con 0x560e9fdc0e20 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9fe38df0 con 0x560e9fdc0500 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.670+0000 7f9df9876b80 10 client.0 _ll_register_callbacks cb 0x7f9dd8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:00.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41760/0 (socket says 172.21.15.145:41760) 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 -- 172.21.15.145:0/3026729411 learned_addr learned my addr 172.21.15.145:0/3026729411 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9ddffff700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fe27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fe1c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 -- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 msgr2=0x560e9fe27760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fe27760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 -- 172.21.15.145:0/3026729411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 msgr2=0x560e9fe1c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fe1c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9dcffff700 1 -- 172.21.15.145:0/3026729411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e9fb38de0 con 0x560e9fdc0500 2023-08-25T04:46:00.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.671+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fe1c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:00.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9dcffff700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4005900 tx=0x7f9dd40042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c0f6cd1414f36b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9dbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:00.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd401e040 con 0x560e9fdc0500 2023-08-25T04:46:00.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9dd400d890 con 0x560e9fdc0500 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e9fdad8b0 con 0x560e9fdc0500 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dd400da50 con 0x560e9fdc0500 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9df9876b80 10 client.25747 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.672+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e9fb21c60 con 0x560e9fdc0500 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e9fdad8b0 con 0x560e9fdc0500 2023-08-25T04:46:00.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9dd4019460 con 0x560e9fdc0500 2023-08-25T04:46:00.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:00.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 10 client.25747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-08-25T04:46:00.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 20 client.25747 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:00.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fe3a4d0 0x560e9fe3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 4 client.25747 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:46:00.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.673+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 --> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e9fd05970 con 0x560e9fe3a4d0 2023-08-25T04:46:00.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fe3a4d0 0x560e9fe3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9dbeffd700 10 client.25747 ms_handle_connect on v2:172.21.15.162:6838/2860621511 2023-08-25T04:46:00.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9ddf7fe700 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fe3a4d0 0x560e9fe3c8f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86e11b0cadeb8d02 server_cookie=929315ca22ad0c48 in_seq=0 out_seq=0 2023-08-25T04:46:00.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9dbeffd700 1 -- 172.21.15.145:0/3026729411 <== mds.1 v2:172.21.15.162:6838/2860621511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e9fd05970 con 0x560e9fe3a4d0 2023-08-25T04:46:00.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9dbeffd700 10 client.25747 handle_command_reply: tid=0 2023-08-25T04:46:00.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 client.25747 shutdown 2023-08-25T04:46:00.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 msgr2=0x560e9fe1cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4005900 tx=0x7f9dd40042e0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 shutdown_connections 2023-08-25T04:46:00.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:6838/2860621511,v1:172.21.15.162:6839/2860621511] conn(0x560e9fe3a4d0 0x560e9fe3c8f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e9fdc0e20 0x560e9fe27760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e9fdbac40 0x560e9fe1c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 --2- 172.21.15.145:0/3026729411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e9fdc0500 0x560e9fe1cc20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 >> 172.21.15.145:0/3026729411 conn(0x560e9fd2f0e0 msgr2=0x560e9fdbdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.674+0000 7f9dbeffd700 0 client.25747 ms_handle_reset on v2:172.21.15.162:6838/2860621511 2023-08-25T04:46:00.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 shutdown_connections 2023-08-25T04:46:00.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9df9876b80 1 -- 172.21.15.145:0/3026729411 wait complete. 2023-08-25T04:46:00.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9df9876b80 20 client.25747 trim_cache size 0 max 16384 2023-08-25T04:46:00.691 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:46:00.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 msgr2=0x7f9dc80a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 0x7f9dc80a2730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9de0069d30 tx=0x7f9dd8009260 comp rx=0 tx=0).stop 2023-08-25T04:46:00.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 msgr2=0x7f9de019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.675+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b070 tx=0x7f9dd000bd90 comp rx=0 tx=0).stop 2023-08-25T04:46:00.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 shutdown_connections 2023-08-25T04:46:00.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de0111720 0x7f9de01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9de0072cc0 0x7f9de019cfc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9dc80a0270 0x7f9dc80a2730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 --2- 172.21.15.145:0/3718125598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de00735e0 0x7f9de019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 >> 172.21.15.145:0/3718125598 conn(0x7f9de00fc1e0 msgr2=0x7f9de00fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 shutdown_connections 2023-08-25T04:46:00.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.676+0000 7f9de4d9d700 1 -- 172.21.15.145:0/3718125598 wait complete. 2023-08-25T04:46:00.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-08-25T04:46:00.839 INFO:teuthology.orchestra.run.smithi145.stderr:15400000 of 50000000 tuples (30%) done (elapsed 38.31 s, remaining 86.07 s) 2023-08-25T04:46:00.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 -- 172.21.15.145:0/591549298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 msgr2=0x7f9830106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 --2- 172.21.15.145:0/591549298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f9830106c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f982c009990 tx=0x7f982c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 -- 172.21.15.145:0/591549298 shutdown_connections 2023-08-25T04:46:00.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 --2- 172.21.15.145:0/591549298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830101250 0x7f983010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.981 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 --2- 172.21.15.145:0/591549298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f9830106c80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 --2- 172.21.15.145:0/591549298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830100890 0x7f9830100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 -- 172.21.15.145:0/591549298 >> 172.21.15.145:0/591549298 conn(0x7f98300fc380 msgr2=0x7f98300fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:00.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 -- 172.21.15.145:0/591549298 shutdown_connections 2023-08-25T04:46:00.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.980+0000 7f98377ff700 1 -- 172.21.15.145:0/591549298 wait complete. 2023-08-25T04:46:00.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.981+0000 7f98377ff700 1 Processor -- start 2023-08-25T04:46:00.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 -- start start 2023-08-25T04:46:00.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830101250 0x7f9830196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f983019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019b930 con 0x7f9830100890 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019baa0 con 0x7f98301068a0 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f98377ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019bc60 con 0x7f9830101250 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f983559b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f983559b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43634/0 (socket says 172.21.15.145:43634) 2023-08-25T04:46:00.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f983559b700 1 -- 172.21.15.145:0/330563036 learned_addr learned my addr 172.21.15.145:0/330563036 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f9834d9a700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830101250 0x7f9830196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.983+0000 7f9835d9c700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f983019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 -- 172.21.15.145:0/330563036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830101250 msgr2=0x7f9830196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830101250 0x7f9830196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 -- 172.21.15.145:0/330563036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 msgr2=0x7f983019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:00.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f983019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 -- 172.21.15.145:0/330563036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0095f0 con 0x7f9830100890 2023-08-25T04:46:00.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.984+0000 7f983559b700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f982c005740 tx=0x7f982c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:00.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.985+0000 7f98267fc700 1 -- 172.21.15.145:0/330563036 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c01d070 con 0x7f9830100890 2023-08-25T04:46:00.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.985+0000 7f98267fc700 1 -- 172.21.15.145:0/330563036 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982c022470 con 0x7f9830100890 2023-08-25T04:46:00.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.985+0000 7f98267fc700 1 -- 172.21.15.145:0/330563036 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c017650 con 0x7f9830100890 2023-08-25T04:46:00.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.985+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983019bee0 con 0x7f9830100890 2023-08-25T04:46:00.989 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.985+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9830072b50 con 0x7f9830100890 2023-08-25T04:46:00.990 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.989+0000 7f98267fc700 1 -- 172.21.15.145:0/330563036 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 23) v1 ==== 134905+0+0 (secure 0 0 0) 0x7f982c0225e0 con 0x7f9830100890 2023-08-25T04:46:00.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.990+0000 7f984b027b80 1 Processor -- start 2023-08-25T04:46:00.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 -- start start 2023-08-25T04:46:00.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4deb7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:00.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4de177b0 con 0x562e4debdb90 2023-08-25T04:46:00.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4de08780 con 0x562e4debe4b0 2023-08-25T04:46:00.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.991+0000 7f984b027b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4de060a0 con 0x562e4deb8450 2023-08-25T04:46:00.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f983559b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4deb7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41778/0 (socket says 172.21.15.145:41778) 2023-08-25T04:46:00.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 -- 172.21.15.145:0/1584614452 learned_addr learned my addr 172.21.15.145:0/1584614452 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:00.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9835d9c700 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:00.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 -- 172.21.15.145:0/1584614452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 msgr2=0x562e4deb7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4deb7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 -- 172.21.15.145:0/1584614452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 msgr2=0x562e4debdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:00.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:00.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 -- 172.21.15.145:0/1584614452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4dc81630 con 0x562e4deb8450 2023-08-25T04:46:01.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9835d9c700 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.992+0000 7f9834d9a700 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9820009cf0 tx=0x7f9820011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc036e38ba79d96 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.993+0000 7f981b7fe700 1 -- 172.21.15.145:0/1584614452 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820011e30 con 0x562e4deb8450 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.993+0000 7f981b7fe700 1 -- 172.21.15.145:0/1584614452 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9820004510 con 0x562e4deb8450 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.993+0000 7f981b7fe700 1 -- 172.21.15.145:0/1584614452 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820005280 con 0x562e4deb8450 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.993+0000 7f98267fc700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 0x7f981c0a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.993+0000 7f98267fc700 1 -- 172.21.15.145:0/330563036 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f982c0cce80 con 0x7f9830100890 2023-08-25T04:46:01.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.994+0000 7f984b027b80 1 -- 172.21.15.145:0/1584614452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 msgr2=0x562e4deb88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.994+0000 7f984b027b80 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9820009cf0 tx=0x7f9820011440 comp rx=0 tx=0).stop 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.994+0000 7f9835d9c700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 0x7f981c0a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f9835d9c700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 0x7f981c0a2720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9828009510 tx=0x7f982800b040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 -- 172.21.15.145:0/1584614452 shutdown_connections 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4deb88b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 --2- 172.21.15.145:0/1584614452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4deb7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 -- 172.21.15.145:0/1584614452 >> 172.21.15.145:0/1584614452 conn(0x562e4de240e0 msgr2=0x562e4deaa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 -- 172.21.15.145:0/1584614452 shutdown_connections 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 -- 172.21.15.145:0/1584614452 wait complete. 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.995+0000 7f984b027b80 1 -- start start 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.996+0000 7f984b027b80 1 Processor -- start 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 0x562e4dec16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debe4b0 0x562e4dec1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4de01990 con 0x562e4debe4b0 2023-08-25T04:46:01.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4dd9fbb0 con 0x562e4deb8450 2023-08-25T04:46:01.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4dd9f8b0 con 0x562e4debdb90 2023-08-25T04:46:01.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f984b027b80 10 client.0 _ll_register_callbacks cb 0x562e4dce9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:01.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f983559b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f983559b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57788/0 (socket says 172.21.15.145:57788) 2023-08-25T04:46:01.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f983559b700 1 -- 172.21.15.145:0/2248749036 learned_addr learned my addr 172.21.15.145:0/2248749036 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:01.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f9835d9c700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debe4b0 0x562e4dec1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.998+0000 7f9834d9a700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 0x562e4dec16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 -- 172.21.15.145:0/2248749036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 msgr2=0x562e4dec16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 0x562e4dec16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 -- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debe4b0 msgr2=0x562e4dec1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debe4b0 0x562e4dec1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 -- 172.21.15.145:0/2248749036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9820009920 con 0x562e4deb8450 2023-08-25T04:46:01.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f9834d9a700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 0x562e4dec16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:01.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f983559b700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f982c09a080 tx=0x7f982c09a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4819a6e5f9a90d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:01.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f9818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:46:01.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a1be0 con 0x562e4deb8450 2023-08-25T04:46:01.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e4de7f1c0 con 0x562e4deb8450 2023-08-25T04:46:01.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f982c09a7f0 con 0x562e4deb8450 2023-08-25T04:46:01.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982c099ec0 con 0x562e4deb8450 2023-08-25T04:46:01.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f984b027b80 10 client.25742 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:01.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:00.999+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e4dc2dde0 con 0x562e4deb8450 2023-08-25T04:46:01.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e4de7f1c0 con 0x562e4deb8450 2023-08-25T04:46:01.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f982c0ca450 con 0x562e4deb8450 2023-08-25T04:46:01.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 10 client.25742 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:01.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:01.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 20 client.25742 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4ded1ad0 0x562e4ded3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 4 client.25742 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e4dda7ba0 con 0x562e4ded1ad0 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.000+0000 7f9834d9a700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4ded1ad0 0x562e4ded3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.001+0000 7f9818ff9700 10 client.25742 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:01.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.001+0000 7f9834d9a700 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4ded1ad0 0x562e4ded3ef0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bdd862529afbea4 server_cookie=fc5cb318e2756539 in_seq=0 out_seq=0 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.002+0000 7f9818ff9700 1 -- 172.21.15.145:0/2248749036 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e4dda7ba0 con 0x562e4ded1ad0 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.002+0000 7f9818ff9700 10 client.25742 handle_command_reply: tid=0 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.002+0000 7f984b027b80 1 client.25742 shutdown 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 msgr2=0x562e4dec4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f982c09a080 tx=0x7f982c09a630 comp rx=0 tx=0).stop 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 shutdown_connections 2023-08-25T04:46:01.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debe4b0 0x562e4dec1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4dec4440 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4ded1ad0 0x562e4ded3ef0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 --2- 172.21.15.145:0/2248749036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debdb90 0x562e4dec16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 >> 172.21.15.145:0/2248749036 conn(0x562e4de240e0 msgr2=0x562e4dea4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:01.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.003+0000 7f9818ff9700 0 client.25742 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:01.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.004+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 shutdown_connections 2023-08-25T04:46:01.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.004+0000 7f984b027b80 1 -- 172.21.15.145:0/2248749036 wait complete. 2023-08-25T04:46:01.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.004+0000 7f984b027b80 20 client.25742 trim_cache size 0 max 16384 2023-08-25T04:46:01.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 Processor -- start 2023-08-25T04:46:01.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 -- start start 2023-08-25T04:46:01.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4dec3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4dc83920 con 0x562e4debdb90 2023-08-25T04:46:01.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4dea6a70 con 0x562e4debe4b0 2023-08-25T04:46:01.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.023+0000 7f984b027b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4dec2670 con 0x562e4deb8450 2023-08-25T04:46:01.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f9835d9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f9835d9c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57790/0 (socket says 172.21.15.145:57790) 2023-08-25T04:46:01.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f9835d9c700 1 -- 172.21.15.145:0/1322279571 learned_addr learned my addr 172.21.15.145:0/1322279571 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:01.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4dec3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f9834d9a700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 -- 172.21.15.145:0/1322279571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 msgr2=0x562e4debbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 -- 172.21.15.145:0/1322279571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 msgr2=0x562e4debb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 -- 172.21.15.145:0/1322279571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4ded8b20 con 0x562e4deb8450 2023-08-25T04:46:01.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f9835d9c700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:46:01.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.024+0000 7f983559b700 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4dec3fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f982c0095c0 tx=0x7f982c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcdec3ddd6c60ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:01.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.025+0000 7f981bfff700 1 -- 172.21.15.145:0/1322279571 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a1ef0 con 0x562e4deb8450 2023-08-25T04:46:01.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.025+0000 7f981bfff700 1 -- 172.21.15.145:0/1322279571 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982c099c30 con 0x562e4deb8450 2023-08-25T04:46:01.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.025+0000 7f981bfff700 1 -- 172.21.15.145:0/1322279571 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a1c40 con 0x562e4deb8450 2023-08-25T04:46:01.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- 172.21.15.145:0/1322279571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 msgr2=0x562e4dec3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4dec3fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f982c0095c0 tx=0x7f982c022c90 comp rx=0 tx=0).stop 2023-08-25T04:46:01.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- 172.21.15.145:0/1322279571 shutdown_connections 2023-08-25T04:46:01.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4debe4b0 0x562e4debbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4deb8450 0x562e4dec3fe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 --2- 172.21.15.145:0/1322279571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4debb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- 172.21.15.145:0/1322279571 >> 172.21.15.145:0/1322279571 conn(0x562e4de240e0 msgr2=0x562e4deaa110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:01.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- 172.21.15.145:0/1322279571 shutdown_connections 2023-08-25T04:46:01.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- 172.21.15.145:0/1322279571 wait complete. 2023-08-25T04:46:01.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.026+0000 7f984b027b80 1 -- start start 2023-08-25T04:46:01.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 Processor -- start 2023-08-25T04:46:01.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4deb9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debe4b0 0x562e4deba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4deba5b0 con 0x562e4debdb90 2023-08-25T04:46:01.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4deba720 con 0x562e4deb8450 2023-08-25T04:46:01.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4deba8e0 con 0x562e4debe4b0 2023-08-25T04:46:01.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.027+0000 7f984b027b80 10 client.0 _ll_register_callbacks cb 0x562e4dce9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:01.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f9834d9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4deb9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57794/0 (socket says 172.21.15.145:57794) 2023-08-25T04:46:01.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 -- 172.21.15.145:0/3267954709 learned_addr learned my addr 172.21.15.145:0/3267954709 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:01.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f9835d9c700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debe4b0 0x562e4deba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 -- 172.21.15.145:0/3267954709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debe4b0 msgr2=0x562e4deba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debe4b0 0x562e4deba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 -- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 msgr2=0x562e4deb9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:01.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4deb9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 -- 172.21.15.145:0/3267954709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828018470 con 0x562e4deb8450 2023-08-25T04:46:01.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f983559b700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f982c099df0 tx=0x7f982c09e060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a5fddc3ce5ffc8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:01.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f9818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:46:01.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.028+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a1ef0 con 0x562e4deb8450 2023-08-25T04:46:01.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e4dea28b0 con 0x562e4deb8450 2023-08-25T04:46:01.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f982c09d070 con 0x562e4deb8450 2023-08-25T04:46:01.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982c0057e0 con 0x562e4deb8450 2023-08-25T04:46:01.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f984b027b80 10 client.25748 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:01.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e4dc2dde0 con 0x562e4deb8450 2023-08-25T04:46:01.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982c07c020 con 0x562e4deb8450 2023-08-25T04:46:01.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.029+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f982c0960e0 con 0x562e4deb8450 2023-08-25T04:46:01.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:01.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:01.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 20 client.25748 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:01.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4df21e30 0x562e4df22210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:01.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 4 client.25748 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-08-25T04:46:01.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e4ddfa970 con 0x562e4df21e30 2023-08-25T04:46:01.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.030+0000 7f9834d9a700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4df21e30 0x562e4df22210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:01.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f9834d9a700 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4df21e30 0x562e4df22210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495df6afca666f15 server_cookie=6c697a7fd02ea073 in_seq=0 out_seq=0 2023-08-25T04:46:01.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f9818ff9700 10 client.25748 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:01.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f9818ff9700 1 -- 172.21.15.145:0/3267954709 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e4ddfa970 con 0x562e4df21e30 2023-08-25T04:46:01.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f9818ff9700 10 client.25748 handle_command_reply: tid=0 2023-08-25T04:46:01.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 client.25748 shutdown 2023-08-25T04:46:01.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 msgr2=0x562e4deb95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f982c099df0 tx=0x7f982c09e060 comp rx=0 tx=0).stop 2023-08-25T04:46:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 shutdown_connections 2023-08-25T04:46:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x562e4df21e30 0x562e4df22210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e4debe4b0 0x562e4deba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e4deb8450 0x562e4deb95b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 --2- 172.21.15.145:0/3267954709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562e4debdb90 0x562e4deb9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 >> 172.21.15.145:0/3267954709 conn(0x562e4de240e0 msgr2=0x562e4debb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.031+0000 7f9818ff9700 0 client.25748 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.032+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 shutdown_connections 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.032+0000 7f984b027b80 1 -- 172.21.15.145:0/3267954709 wait complete. 2023-08-25T04:46:01.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.032+0000 7f984b027b80 20 client.25748 trim_cache size 0 max 16384 2023-08-25T04:46:01.053 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2023-08-25T04:46:01.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 msgr2=0x7f981c0a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 0x7f981c0a2720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9828009510 tx=0x7f982800b040 comp rx=0 tx=0).stop 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 msgr2=0x7f9830196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f982c005740 tx=0x7f982c005770 comp rx=0 tx=0).stop 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 shutdown_connections 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f981c0a0260 0x7f981c0a2720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98301068a0 0x7f983019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9830100890 0x7f9830196990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 --2- 172.21.15.145:0/330563036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9830101250 0x7f9830196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 >> 172.21.15.145:0/330563036 conn(0x7f98300fc380 msgr2=0x7f98300fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 shutdown_connections 2023-08-25T04:46:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:01.033+0000 7f98377ff700 1 -- 172.21.15.145:0/330563036 wait complete. 2023-08-25T04:46:01.055 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:00 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/891706679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:46:01.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:00 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/891706679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:46:01.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:00 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/891706679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T04:46:02.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-08-25T04:46:02.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-08-25T04:46:02.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:01 smithi145 ceph-mon[97681]: pgmap v31: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 143 MiB/s wr, 199 op/s 2023-08-25T04:46:02.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:01 smithi145 ceph-mon[97681]: mgrmap e23: x(active, since 56s), standbys: y, z 2023-08-25T04:46:02.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:01 smithi162 ceph-mon[130941]: pgmap v31: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 143 MiB/s wr, 199 op/s 2023-08-25T04:46:02.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:01 smithi162 ceph-mon[130941]: mgrmap e23: x(active, since 56s), standbys: y, z 2023-08-25T04:46:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:01 smithi183 ceph-mon[131414]: pgmap v31: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 143 MiB/s wr, 199 op/s 2023-08-25T04:46:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:01 smithi183 ceph-mon[131414]: mgrmap e23: x(active, since 56s), standbys: y, z 2023-08-25T04:46:02.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/537874878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b081068a0 msgr2=0x7f9b08106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/537874878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b081068a0 0x7f9b08106c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9af80098f0 tx=0x7f9af8009c00 comp rx=0 tx=0).stop 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/537874878 shutdown_connections 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/537874878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08101250 0x7f9b0810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/537874878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b081068a0 0x7f9b08106c80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/537874878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b08100890 0x7f9b08100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.282+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/537874878 >> 172.21.15.145:0/537874878 conn(0x7f9b080fc380 msgr2=0x7f9b080fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.283+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/537874878 shutdown_connections 2023-08-25T04:46:02.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.283+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/537874878 wait complete. 2023-08-25T04:46:02.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.284+0000 7f9b0c9ff700 1 Processor -- start 2023-08-25T04:46:02.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.284+0000 7f9b0c9ff700 1 -- start start 2023-08-25T04:46:02.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.284+0000 7f9b0c9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.284+0000 7f9b0c9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 0x7f9b0819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0c9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b081068a0 0x7f9b08196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0c9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08196e90 con 0x7f9b08101250 2023-08-25T04:46:02.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0c9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08197000 con 0x7f9b081068a0 2023-08-25T04:46:02.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0c9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b081971c0 con 0x7f9b08100890 2023-08-25T04:46:02.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41830/0 (socket says 172.21.15.145:41830) 2023-08-25T04:46:02.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b081068a0 0x7f9b08196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 -- 172.21.15.145:0/4156993554 learned_addr learned my addr 172.21.15.145:0/4156993554 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 0x7f9b0819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 -- 172.21.15.145:0/4156993554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b081068a0 msgr2=0x7f9b08196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b081068a0 0x7f9b08196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 -- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 msgr2=0x7f9b0819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 0x7f9b0819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b0659c700 1 -- 172.21.15.145:0/4156993554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009520 con 0x7f9b08100890 2023-08-25T04:46:02.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.285+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 0x7f9b0819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:02.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9b0659c700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009810 tx=0x7f9af800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9af77fe700 1 -- 172.21.15.145:0/4156993554 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af801c070 con 0x7f9b08100890 2023-08-25T04:46:02.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b08197440 con 0x7f9b08100890 2023-08-25T04:46:02.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9af77fe700 1 -- 172.21.15.145:0/4156993554 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af8005040 con 0x7f9b08100890 2023-08-25T04:46:02.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9af77fe700 1 -- 172.21.15.145:0/4156993554 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8004a30 con 0x7f9b08100890 2023-08-25T04:46:02.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.286+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b081a37a0 con 0x7f9b08100890 2023-08-25T04:46:02.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.290+0000 7f9af77fe700 1 -- 172.21.15.145:0/4156993554 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) v1 ==== 134905+0+0 (secure 0 0 0) 0x7f9af8020070 con 0x7f9b08100890 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.292+0000 7f9af77fe700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 0x7f9af00a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.292+0000 7f9af77fe700 1 -- 172.21.15.145:0/4156993554 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9af80cc950 con 0x7f9b08100890 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.292+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 0x7f9af00a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.293+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 0x7f9af00a26a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005950 tx=0x7f9afc0058e0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.293+0000 7f9b2031bb80 1 Processor -- start 2023-08-25T04:46:02.299 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.293+0000 7f9b2031bb80 1 -- start start 2023-08-25T04:46:02.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e875cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 0x55e0e8756b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.300 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8757510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e86ba830 con 0x55e0e87570d0 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e86ab800 con 0x55e0e875d0f0 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e86a9120 con 0x55e0e875c7d0 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e875cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 0x55e0e8756b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.301 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e875cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41834/0 (socket says 172.21.15.145:41834) 2023-08-25T04:46:02.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b0659c700 1 -- 172.21.15.145:0/1315891197 learned_addr learned my addr 172.21.15.145:0/1315891197 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8757510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 -- 172.21.15.145:0/1315891197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 msgr2=0x55e0e875cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e875cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 -- 172.21.15.145:0/1315891197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 msgr2=0x55e0e8757510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8757510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.294+0000 7f9b05d9b700 1 -- 172.21.15.145:0/1315891197 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e85246b0 con 0x55e0e875d0f0 2023-08-25T04:46:02.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.295+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 0x55e0e8756b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b080fecf0 tx=0x7f9afc012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ab6dc114bb46e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.295+0000 7f9af4ff9700 1 -- 172.21.15.145:0/1315891197 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc011a50 con 0x55e0e875d0f0 2023-08-25T04:46:02.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.295+0000 7f9af4ff9700 1 -- 172.21.15.145:0/1315891197 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9afc011bb0 con 0x55e0e875d0f0 2023-08-25T04:46:02.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.295+0000 7f9af4ff9700 1 -- 172.21.15.145:0/1315891197 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc01d610 con 0x55e0e875d0f0 2023-08-25T04:46:02.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 -- 172.21.15.145:0/1315891197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 msgr2=0x55e0e8756b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 0x55e0e8756b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b080fecf0 tx=0x7f9afc012000 comp rx=0 tx=0).stop 2023-08-25T04:46:02.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 -- 172.21.15.145:0/1315891197 shutdown_connections 2023-08-25T04:46:02.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8757510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e875cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/1315891197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875d0f0 0x55e0e8756b90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 -- 172.21.15.145:0/1315891197 >> 172.21.15.145:0/1315891197 conn(0x55e0e86c7160 msgr2=0x55e0e874d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 -- 172.21.15.145:0/1315891197 shutdown_connections 2023-08-25T04:46:02.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.296+0000 7f9b2031bb80 1 -- 172.21.15.145:0/1315891197 wait complete. 2023-08-25T04:46:02.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.297+0000 7f9b2031bb80 1 -- start start 2023-08-25T04:46:02.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.297+0000 7f9b2031bb80 1 Processor -- start 2023-08-25T04:46:02.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e87602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 0x55e0e87608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e86a4a10 con 0x55e0e875d0f0 2023-08-25T04:46:02.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e8642c30 con 0x55e0e87570d0 2023-08-25T04:46:02.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e8642930 con 0x55e0e875c7d0 2023-08-25T04:46:02.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b2031bb80 10 client.0 _ll_register_callbacks cb 0x55e0e858cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:02.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57824/0 (socket says 172.21.15.145:57824) 2023-08-25T04:46:02.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b0659c700 1 -- 172.21.15.145:0/3373985439 learned_addr learned my addr 172.21.15.145:0/3373985439 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.298+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e87602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 0x55e0e87608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 -- 172.21.15.145:0/3373985439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 msgr2=0x55e0e87602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e87602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 -- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 msgr2=0x55e0e87608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 0x55e0e87608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 -- 172.21.15.145:0/3373985439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc00bdf0 con 0x55e0e87570d0 2023-08-25T04:46:02.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 0x55e0e87608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b0659c700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ca3c0 tx=0x7f9af8099bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc83ab031e9b922 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af809a020 con 0x55e0e87570d0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9af80a1940 con 0x55e0e87570d0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0e8722240 con 0x55e0e87570d0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.299+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af80056c0 con 0x55e0e87570d0 2023-08-25T04:46:02.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 10 client.25760 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:02.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0e84d0e60 con 0x55e0e87570d0 2023-08-25T04:46:02.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af8063020 con 0x55e0e87570d0 2023-08-25T04:46:02.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9af80a14b0 con 0x55e0e87570d0 2023-08-25T04:46:02.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:02.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:02.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 20 client.25760 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:02.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e8770860 0x55e0e8772c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 4 client.25760 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:02.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0e864ac20 con 0x55e0e8770860 2023-08-25T04:46:02.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.300+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e8770860 0x55e0e8772c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.301+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e8770860 0x55e0e8772c80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146fa3d2891a68f6 server_cookie=58ecb3b0d460e5fa in_seq=0 out_seq=0 2023-08-25T04:46:02.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.301+0000 7f9aedffb700 10 client.25760 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9aedffb700 1 -- 172.21.15.145:0/3373985439 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0e864ac20 con 0x55e0e8770860 2023-08-25T04:46:02.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9aedffb700 10 client.25760 handle_command_reply: tid=0 2023-08-25T04:46:02.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 client.25760 shutdown 2023-08-25T04:46:02.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 msgr2=0x55e0e87630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ca3c0 tx=0x7f9af8099bc0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 shutdown_connections 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875d0f0 0x55e0e87608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e87630a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e8770860 0x55e0e8772c80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/3373985439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875c7d0 0x55e0e87602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 >> 172.21.15.145:0/3373985439 conn(0x55e0e86c7160 msgr2=0x55e0e8747330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9aedffb700 0 client.25760 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 shutdown_connections 2023-08-25T04:46:02.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.302+0000 7f9b2031bb80 1 -- 172.21.15.145:0/3373985439 wait complete. 2023-08-25T04:46:02.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.303+0000 7f9b2031bb80 20 client.25760 trim_cache size 0 max 16384 2023-08-25T04:46:02.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 Processor -- start 2023-08-25T04:46:02.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 -- start start 2023-08-25T04:46:02.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 0x55e0e875fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8760090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e85269a0 con 0x55e0e875c7d0 2023-08-25T04:46:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e87456c0 con 0x55e0e87570d0 2023-08-25T04:46:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.323+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e875ada0 con 0x55e0e875d0f0 2023-08-25T04:46:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57840/0 (socket says 172.21.15.145:57840) 2023-08-25T04:46:02.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 -- 172.21.15.145:0/822198044 learned_addr learned my addr 172.21.15.145:0/822198044 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8760090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 0x55e0e875fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 -- 172.21.15.145:0/822198044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 msgr2=0x55e0e8760090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8760090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 -- 172.21.15.145:0/822198044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 msgr2=0x55e0e875fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 0x55e0e875fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 -- 172.21.15.145:0/822198044 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e87778b0 con 0x55e0e87570d0 2023-08-25T04:46:02.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8760090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:02.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 0x55e0e875fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:02.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9b0659c700 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005910 tx=0x7f9af8099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31533be159b46994 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9af57fa700 1 -- 172.21.15.145:0/822198044 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80056c0 con 0x55e0e87570d0 2023-08-25T04:46:02.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9af57fa700 1 -- 172.21.15.145:0/822198044 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af8063070 con 0x55e0e87570d0 2023-08-25T04:46:02.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.324+0000 7f9af57fa700 1 -- 172.21.15.145:0/822198044 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80992b0 con 0x55e0e87570d0 2023-08-25T04:46:02.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- 172.21.15.145:0/822198044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 msgr2=0x55e0e8762bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005910 tx=0x7f9af8099ac0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- 172.21.15.145:0/822198044 shutdown_connections 2023-08-25T04:46:02.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8760090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e87570d0 0x55e0e8762bd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/822198044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e875c7d0 0x55e0e875fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- 172.21.15.145:0/822198044 >> 172.21.15.145:0/822198044 conn(0x55e0e86c7160 msgr2=0x55e0e874d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- 172.21.15.145:0/822198044 shutdown_connections 2023-08-25T04:46:02.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- 172.21.15.145:0/822198044 wait complete. 2023-08-25T04:46:02.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 -- start start 2023-08-25T04:46:02.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.326+0000 7f9b2031bb80 1 Processor -- start 2023-08-25T04:46:02.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8767cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e8758cb0 con 0x55e0e87570d0 2023-08-25T04:46:02.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e8758e20 con 0x55e0e875c7d0 2023-08-25T04:46:02.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e8758f90 con 0x55e0e875d0f0 2023-08-25T04:46:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b2031bb80 10 client.0 _ll_register_callbacks cb 0x55e0e858cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b05d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b06d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b05d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57844/0 (socket says 172.21.15.145:57844) 2023-08-25T04:46:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b06d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41856/0 (socket says 172.21.15.145:41856) 2023-08-25T04:46:02.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.327+0000 7f9b05d9b700 1 -- 172.21.15.145:0/261728701 learned_addr learned my addr 172.21.15.145:0/261728701 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b0659c700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8767cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 -- 172.21.15.145:0/261728701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 msgr2=0x55e0e8758770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 -- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 msgr2=0x55e0e8767cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8767cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 -- 172.21.15.145:0/261728701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0e84d0e60 con 0x55e0e875c7d0 2023-08-25T04:46:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b06d9d700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:46:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b05d9b700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9afc014040 tx=0x7f9afc01e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8252ed241b80a6d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:46:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc02d040 con 0x55e0e875c7d0 2023-08-25T04:46:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0e8745930 con 0x55e0e875c7d0 2023-08-25T04:46:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f9afc01e790 con 0x55e0e875c7d0 2023-08-25T04:46:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9afc01e950 con 0x55e0e875c7d0 2023-08-25T04:46:02.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.328+0000 7f9b2031bb80 10 client.25766 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:02.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0e84b9ce0 con 0x55e0e875c7d0 2023-08-25T04:46:02.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0e8745930 con 0x55e0e875c7d0 2023-08-25T04:46:02.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9afc011ae0 con 0x55e0e875c7d0 2023-08-25T04:46:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 20 client.25766 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:02.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e87c5b10 0x55e0e87c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 4 client.25766 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-08-25T04:46:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e0e869d9f0 con 0x55e0e87c5b10 2023-08-25T04:46:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.329+0000 7f9b0659c700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e87c5b10 0x55e0e87c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.330+0000 7f9b0659c700 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e87c5b10 0x55e0e87c7f30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2221834203db5d63 server_cookie=186b7ff6118efed in_seq=0 out_seq=0 2023-08-25T04:46:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.330+0000 7f9aedffb700 10 client.25766 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.332+0000 7f9aedffb700 1 -- 172.21.15.145:0/261728701 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e0e869d9f0 con 0x55e0e87c5b10 2023-08-25T04:46:02.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.332+0000 7f9aedffb700 10 client.25766 handle_command_reply: tid=0 2023-08-25T04:46:02.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.332+0000 7f9b2031bb80 1 client.25766 shutdown 2023-08-25T04:46:02.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.332+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 msgr2=0x55e0e8758230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.332+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9afc014040 tx=0x7f9afc01e5a0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 shutdown_connections 2023-08-25T04:46:02.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e0e87c5b10 0x55e0e87c7f30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0e875d0f0 0x55e0e8758770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e0e87570d0 0x55e0e8767cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 --2- 172.21.15.145:0/261728701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e0e875c7d0 0x55e0e8758230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9aedffb700 0 client.25766 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 >> 172.21.15.145:0/261728701 conn(0x55e0e86c7160 msgr2=0x55e0e875fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 shutdown_connections 2023-08-25T04:46:02.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.333+0000 7f9b2031bb80 1 -- 172.21.15.145:0/261728701 wait complete. 2023-08-25T04:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:46:02.365 INFO:teuthology.orchestra.run.smithi145.stdout: "return_code": 0, 2023-08-25T04:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": "asynchronous" 2023-08-25T04:46:02.366 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:46:02.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.334+0000 7f9b2031bb80 20 client.25766 trim_cache size 0 max 16384 2023-08-25T04:46:02.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 msgr2=0x7f9af00a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 0x7f9af00a26a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005950 tx=0x7f9afc0058e0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 msgr2=0x7f9b0819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009810 tx=0x7f9af800fa70 comp rx=0 tx=0).stop 2023-08-25T04:46:02.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 shutdown_connections 2023-08-25T04:46:02.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9af00a01e0 0x7f9af00a26a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b081068a0 0x7f9b08196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.336+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b08100890 0x7f9b0819c7b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.337+0000 7f9b0c9ff700 1 --2- 172.21.15.145:0/4156993554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b08101250 0x7f9b0819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.337+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 >> 172.21.15.145:0/4156993554 conn(0x7f9b080fc380 msgr2=0x7f9b081043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.337+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 shutdown_connections 2023-08-25T04:46:02.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.337+0000 7f9b0c9ff700 1 -- 172.21.15.145:0/4156993554 wait complete. 2023-08-25T04:46:02.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:46:02.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 -- 172.21.15.145:0/1240050077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe940072cc0 msgr2=0x7fe9400730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 --2- 172.21.15.145:0/1240050077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe940072cc0 0x7fe9400730a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe930009990 tx=0x7fe930009ca0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 -- 172.21.15.145:0/1240050077 shutdown_connections 2023-08-25T04:46:02.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 --2- 172.21.15.145:0/1240050077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe940113c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 --2- 172.21.15.145:0/1240050077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe940072cc0 0x7fe9400730a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 --2- 172.21.15.145:0/1240050077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9400735e0 0x7fe940111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 -- 172.21.15.145:0/1240050077 >> 172.21.15.145:0/1240050077 conn(0x7fe9400fc3a0 msgr2=0x7fe9400fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.691+0000 7fe944bff700 1 -- 172.21.15.145:0/1240050077 shutdown_connections 2023-08-25T04:46:02.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.692+0000 7fe944bff700 1 -- 172.21.15.145:0/1240050077 wait complete. 2023-08-25T04:46:02.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.693+0000 7fe944bff700 1 Processor -- start 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.693+0000 7fe944bff700 1 -- start start 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe940072cc0 0x7fe94019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe9401a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401a1ff0 con 0x7fe940072cc0 2023-08-25T04:46:02.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401a2160 con 0x7fe9400735e0 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe944bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9401a2320 con 0x7fe940111880 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57868/0 (socket says 172.21.15.145:57868) 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3035484340 learned_addr learned my addr 172.21.15.145:0/3035484340 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe9401a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3035484340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 msgr2=0x7fe9401a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe9401a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe940072cc0 msgr2=0x7fe94019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe940072cc0 0x7fe94019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3035484340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9300095f0 con 0x7fe9400735e0 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.694+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe9401a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e9a0 tx=0x7fe93400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe92f7fe700 1 -- 172.21.15.145:0/3035484340 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934009790 con 0x7fe9400735e0 2023-08-25T04:46:02.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe92f7fe700 1 -- 172.21.15.145:0/3035484340 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe93400cbd0 con 0x7fe9400735e0 2023-08-25T04:46:02.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9401a2600 con 0x7fe9400735e0 2023-08-25T04:46:02.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe92f7fe700 1 -- 172.21.15.145:0/3035484340 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934018680 con 0x7fe9400735e0 2023-08-25T04:46:02.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.695+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9401a9e40 con 0x7fe9400735e0 2023-08-25T04:46:02.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.701+0000 7fe958492b80 1 Processor -- start 2023-08-25T04:46:02.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.701+0000 7fe92f7fe700 1 -- 172.21.15.145:0/3035484340 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 23) v1 ==== 134905+0+0 (secure 0 0 0) 0x7fe93400cd40 con 0x7fe9400735e0 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.701+0000 7fe958492b80 1 -- start start 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.701+0000 7fe958492b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.701+0000 7fe958492b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe958492b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe958492b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c51327b0 con 0x5654c51d0ca0 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe958492b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c5123780 con 0x5654c51cac40 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe958492b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c51210a0 con 0x5654c51d0380 2023-08-25T04:46:02.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57882/0 (socket says 172.21.15.145:57882) 2023-08-25T04:46:02.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3349400064 learned_addr learned my addr 172.21.15.145:0/3349400064 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93e59c700 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3349400064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 msgr2=0x5654c51d0760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T04:46:02.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3349400064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 msgr2=0x5654c51ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.702+0000 7fe93dd9b700 1 -- 172.21.15.145:0/3349400064 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c4f9c630 con 0x5654c51cac40 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.703+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9340c9100 tx=0x7fe9340d43b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a2a363289f8864 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.703+0000 7fe92cff9700 1 -- 172.21.15.145:0/3349400064 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340de470 con 0x5654c51cac40 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.703+0000 7fe92cff9700 1 -- 172.21.15.145:0/3349400064 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9340dea90 con 0x5654c51cac40 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.703+0000 7fe92cff9700 1 -- 172.21.15.145:0/3349400064 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340d2950 con 0x5654c51cac40 2023-08-25T04:46:02.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe92f7fe700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 0x7fe9280a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe92f7fe700 1 -- 172.21.15.145:0/3035484340 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe934014070 con 0x7fe9400735e0 2023-08-25T04:46:02.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 0x7fe9280a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 -- 172.21.15.145:0/3349400064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 msgr2=0x5654c51cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9340c9100 tx=0x7fe9340d43b0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 -- 172.21.15.145:0/3349400064 shutdown_connections 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51cb0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 --2- 172.21.15.145:0/3349400064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.704+0000 7fe958492b80 1 -- 172.21.15.145:0/3349400064 >> 172.21.15.145:0/3349400064 conn(0x5654c513f0e0 msgr2=0x5654c51c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.705+0000 7fe958492b80 1 -- 172.21.15.145:0/3349400064 shutdown_connections 2023-08-25T04:46:02.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.705+0000 7fe958492b80 1 -- 172.21.15.145:0/3349400064 wait complete. 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.705+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 0x7fe9280a2720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fd0 tx=0x7fe93800b040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.705+0000 7fe958492b80 1 -- start start 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 Processor -- start 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c511c990 con 0x5654c51d0ca0 2023-08-25T04:46:02.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c50babb0 con 0x5654c51d0380 2023-08-25T04:46:02.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c50ba8b0 con 0x5654c51cac40 2023-08-25T04:46:02.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.706+0000 7fe958492b80 10 client.0 _ll_register_callbacks cb 0x5654c5004a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:02.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57894/0 (socket says 172.21.15.145:57894) 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 -- 172.21.15.145:0/2592284137 learned_addr learned my addr 172.21.15.145:0/2592284137 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 -- 172.21.15.145:0/2592284137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 msgr2=0x5654c51d6c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d6c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 -- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 msgr2=0x5654c51d4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51d4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 -- 172.21.15.145:0/2592284137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9401a9e40 con 0x5654c51d0380 2023-08-25T04:46:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.707+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e970 tx=0x7fe9340cf540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb84c829a6d9978 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.708+0000 7fe925ffb700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:46:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.709+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934095390 con 0x5654c51d0380 2023-08-25T04:46:02.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.709+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe934099750 con 0x5654c51d0380 2023-08-25T04:46:02.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.709+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654c519a1c0 con 0x5654c51d0380 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe934098e20 con 0x5654c51d0380 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 10 client.25778 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5654c4f48de0 con 0x5654c51d0380 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654c519a1c0 con 0x5654c51d0380 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe9340992b0 con 0x5654c51d0380 2023-08-25T04:46:02.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 20 client.25778 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c51e4260 0x5654c51e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 4 client.25778 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.710+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654c50c2ba0 con 0x5654c51e4260 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.711+0000 7fe93e59c700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c51e4260 0x5654c51e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.711+0000 7fe925ffb700 10 client.25778 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.711+0000 7fe93e59c700 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c51e4260 0x5654c51e6650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9279ad069ab8b92b server_cookie=2aa8bdc02de0c88e in_seq=0 out_seq=0 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.712+0000 7fe925ffb700 1 -- 172.21.15.145:0/2592284137 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5654c50c2ba0 con 0x5654c51e4260 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.712+0000 7fe925ffb700 10 client.25778 handle_command_reply: tid=0 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.712+0000 7fe958492b80 1 client.25778 shutdown 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.713+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 msgr2=0x5654c51d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.713+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e970 tx=0x7fe9340cf540 comp rx=0 tx=0).stop 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 shutdown_connections 2023-08-25T04:46:02.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c51e4260 0x5654c51e6650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 --2- 172.21.15.145:0/2592284137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0380 0x5654c51d3ee0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 >> 172.21.15.145:0/2592284137 conn(0x5654c513f0e0 msgr2=0x5654c51bfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe925ffb700 0 client.25778 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 shutdown_connections 2023-08-25T04:46:02.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 1 -- 172.21.15.145:0/2592284137 wait complete. 2023-08-25T04:46:02.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.714+0000 7fe958492b80 20 client.25778 trim_cache size 0 max 16384 2023-08-25T04:46:02.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 Processor -- start 2023-08-25T04:46:02.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 -- start start 2023-08-25T04:46:02.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.737+0000 7fe958492b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4f9e920 con 0x5654c51d0ca0 2023-08-25T04:46:02.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe958492b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c51cb600 con 0x5654c51cac40 2023-08-25T04:46:02.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe958492b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c51c1c80 con 0x5654c51d0380 2023-08-25T04:46:02.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57910/0 (socket says 172.21.15.145:57910) 2023-08-25T04:46:02.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 -- 172.21.15.145:0/627371647 learned_addr learned my addr 172.21.15.145:0/627371647 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93ed9d700 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 -- 172.21.15.145:0/627371647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 msgr2=0x5654c51d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 -- 172.21.15.145:0/627371647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 msgr2=0x5654c51cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.738+0000 7fe93e59c700 1 -- 172.21.15.145:0/627371647 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c4f31c60 con 0x5654c51cac40 2023-08-25T04:46:02.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.739+0000 7fe93e59c700 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe930009990 tx=0x7fe93000b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b829b4269692aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.739+0000 7fe92d7fa700 1 -- 172.21.15.145:0/627371647 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930025070 con 0x5654c51cac40 2023-08-25T04:46:02.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.739+0000 7fe92d7fa700 1 -- 172.21.15.145:0/627371647 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe93001e040 con 0x5654c51cac40 2023-08-25T04:46:02.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.739+0000 7fe92d7fa700 1 -- 172.21.15.145:0/627371647 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000ba10 con 0x5654c51cac40 2023-08-25T04:46:02.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- 172.21.15.145:0/627371647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 msgr2=0x5654c51db7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe930009990 tx=0x7fe93000b610 comp rx=0 tx=0).stop 2023-08-25T04:46:02.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- 172.21.15.145:0/627371647 shutdown_connections 2023-08-25T04:46:02.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0ca0 0x5654c51cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51cac40 0x5654c51db7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 --2- 172.21.15.145:0/627371647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51d0380 0x5654c51d6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- 172.21.15.145:0/627371647 >> 172.21.15.145:0/627371647 conn(0x5654c513f0e0 msgr2=0x5654c51ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- 172.21.15.145:0/627371647 shutdown_connections 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- 172.21.15.145:0/627371647 wait complete. 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.740+0000 7fe958492b80 1 -- start start 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.741+0000 7fe958492b80 1 Processor -- start 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.741+0000 7fe958492b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.741+0000 7fe958492b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 0x5654c51d9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.741+0000 7fe958492b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0ca0 0x5654c5237080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe958492b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c51da600 con 0x5654c51d0380 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe958492b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c52375c0 con 0x5654c51d0ca0 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe958492b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c5237780 con 0x5654c51cac40 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe958492b80 10 client.0 _ll_register_callbacks cb 0x5654c5004a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41920/0 (socket says 172.21.15.145:41920) 2023-08-25T04:46:02.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0ca0 0x5654c5237080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 -- 172.21.15.145:0/1528704134 learned_addr learned my addr 172.21.15.145:0/1528704134 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 0x5654c51d9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 -- 172.21.15.145:0/1528704134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0ca0 msgr2=0x5654c5237080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0ca0 0x5654c5237080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 -- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 msgr2=0x5654c51d9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 0x5654c51d9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 -- 172.21.15.145:0/1528704134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe940114620 con 0x5654c51cac40 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 0x5654c51d9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe93e59c700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe930005f50 tx=0x7fe9300053a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93bab0803c4b29b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.742+0000 7fe925ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930005550 con 0x5654c51cac40 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fe9300056b0 con 0x5654c51cac40 2023-08-25T04:46:02.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654c51bd8b0 con 0x5654c51cac40 2023-08-25T04:46:02.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe930025050 con 0x5654c51cac40 2023-08-25T04:46:02.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe958492b80 10 client.25792 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:02.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5654c4f6f250 con 0x5654c51cac40 2023-08-25T04:46:02.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.743+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654c51bd8b0 con 0x5654c51cac40 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe9300046a0 con 0x5654c51cac40 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 20 client.25792 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c5238bb0 0x5654c523afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:02.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 4 client.25792 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5654c5115970 con 0x5654c5238bb0 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.744+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c5238bb0 0x5654c523afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe93dd9b700 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c5238bb0 0x5654c523afd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0aba0a564d4e958 server_cookie=6db949c1084ecd95 in_seq=0 out_seq=0 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe925ffb700 10 client.25792 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe925ffb700 1 -- 172.21.15.145:0/1528704134 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5654c5115970 con 0x5654c5238bb0 2023-08-25T04:46:02.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe925ffb700 10 client.25792 handle_command_reply: tid=0 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 client.25792 shutdown 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 msgr2=0x5654c51d9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe930005f50 tx=0x7fe9300053a0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 shutdown_connections 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5654c51d0ca0 0x5654c5237080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5654c5238bb0 0x5654c523afd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654c51cac40 0x5654c51d9970 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 --2- 172.21.15.145:0/1528704134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654c51d0380 0x5654c51d9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 >> 172.21.15.145:0/1528704134 conn(0x5654c513f0e0 msgr2=0x5654c51bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.745+0000 7fe925ffb700 0 client.25792 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:02.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.746+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 shutdown_connections 2023-08-25T04:46:02.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.746+0000 7fe958492b80 1 -- 172.21.15.145:0/1528704134 wait complete. 2023-08-25T04:46:02.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.746+0000 7fe958492b80 20 client.25792 trim_cache size 0 max 16384 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:46:02.756 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:46:02.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:46:02.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:46:02.757 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:46:02.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 msgr2=0x7fe9280a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 0x7fe9280a2720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fd0 tx=0x7fe93800b040 comp rx=0 tx=0).stop 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 msgr2=0x7fe94019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e9a0 tx=0x7fe93400ed60 comp rx=0 tx=0).stop 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 shutdown_connections 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe940111880 0x7fe9401a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe940072cc0 0x7fe94019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fe9280a0260 0x7fe9280a2720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 --2- 172.21.15.145:0/3035484340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9400735e0 0x7fe94019d550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:02.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 >> 172.21.15.145:0/3035484340 conn(0x7fe9400fc3a0 msgr2=0x7fe9400fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:02.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 shutdown_connections 2023-08-25T04:46:02.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:02.747+0000 7fe944bff700 1 -- 172.21.15.145:0/3035484340 wait complete. 2023-08-25T04:46:02.769 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub queued for path: / 2023-08-25T04:46:02.769 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:02.769 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:46:02.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:46:02.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:02.770 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:46:02.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:02.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:46:02.771 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:02 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:02.773 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:46:03.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub queued for path: / 2023-08-25T04:46:03.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:46:03.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:46:03.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:46:03.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:46:03.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:02 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub queued for path: / 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:46:03.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:03.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:46:03.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:02 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:04.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:03 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:46:04.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:03 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:04.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:03 smithi145 ceph-mon[97681]: pgmap v32: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:03 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:46:04.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:03 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:04.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:03 smithi162 ceph-mon[130941]: pgmap v32: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:03 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:46:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:03 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:46:04.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:03 smithi183 ceph-mon[131414]: pgmap v32: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:05 smithi145 ceph-mon[97681]: pgmap v33: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:05 smithi162 ceph-mon[130941]: pgmap v33: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:05 smithi183 ceph-mon[131414]: pgmap v33: 129 pgs: 129 active+clean; 2.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 MiB/s wr, 164 op/s 2023-08-25T04:46:08.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:07 smithi145 ceph-mon[97681]: pgmap v34: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 316 op/s 2023-08-25T04:46:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:07 smithi162 ceph-mon[130941]: pgmap v34: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 316 op/s 2023-08-25T04:46:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:07 smithi183 ceph-mon[131414]: pgmap v34: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 77 MiB/s wr, 316 op/s 2023-08-25T04:46:10.035 INFO:teuthology.orchestra.run.smithi145.stderr:15500000 of 50000000 tuples (31%) done (elapsed 47.51 s, remaining 105.74 s) 2023-08-25T04:46:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:09 smithi145 ceph-mon[97681]: pgmap v35: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 292 op/s 2023-08-25T04:46:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:09 smithi162 ceph-mon[130941]: pgmap v35: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 292 op/s 2023-08-25T04:46:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:09 smithi183 ceph-mon[131414]: pgmap v35: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 292 op/s 2023-08-25T04:46:12.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:11 smithi145 ceph-mon[97681]: pgmap v36: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 439 op/s 2023-08-25T04:46:12.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:11 smithi162 ceph-mon[130941]: pgmap v36: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 439 op/s 2023-08-25T04:46:12.165 INFO:teuthology.orchestra.run.smithi145.stderr:15600000 of 50000000 tuples (31%) done (elapsed 49.63 s, remaining 109.45 s) 2023-08-25T04:46:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:11 smithi183 ceph-mon[131414]: pgmap v36: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 439 op/s 2023-08-25T04:46:12.369 INFO:teuthology.orchestra.run.smithi145.stderr:15700000 of 50000000 tuples (31%) done (elapsed 49.84 s, remaining 108.88 s) 2023-08-25T04:46:12.831 INFO:teuthology.orchestra.run.smithi145.stderr:15800000 of 50000000 tuples (31%) done (elapsed 50.05 s, remaining 108.33 s) 2023-08-25T04:46:12.831 INFO:teuthology.orchestra.run.smithi145.stderr:15900000 of 50000000 tuples (31%) done (elapsed 50.25 s, remaining 107.78 s) 2023-08-25T04:46:13.059 INFO:teuthology.orchestra.run.smithi145.stderr:16000000 of 50000000 tuples (32%) done (elapsed 50.53 s, remaining 107.37 s) 2023-08-25T04:46:13.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:12 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:46:13.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:12 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:46:13.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:12 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:46:13.283 INFO:teuthology.orchestra.run.smithi145.stderr:16100000 of 50000000 tuples (32%) done (elapsed 50.75 s, remaining 106.86 s) 2023-08-25T04:46:13.958 INFO:teuthology.orchestra.run.smithi145.stderr:16200000 of 50000000 tuples (32%) done (elapsed 51.43 s, remaining 107.30 s) 2023-08-25T04:46:14.084 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:13 smithi145 ceph-mon[97681]: pgmap v37: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:13 smithi162 ceph-mon[130941]: pgmap v37: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:13 smithi183 ceph-mon[131414]: pgmap v37: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:14.310 INFO:teuthology.orchestra.run.smithi145.stderr:16300000 of 50000000 tuples (32%) done (elapsed 51.78 s, remaining 107.05 s) 2023-08-25T04:46:14.627 INFO:teuthology.orchestra.run.smithi145.stderr:16400000 of 50000000 tuples (32%) done (elapsed 52.10 s, remaining 106.73 s) 2023-08-25T04:46:14.824 INFO:teuthology.orchestra.run.smithi145.stderr:16500000 of 50000000 tuples (33%) done (elapsed 52.29 s, remaining 106.17 s) 2023-08-25T04:46:15.001 INFO:teuthology.orchestra.run.smithi145.stderr:16600000 of 50000000 tuples (33%) done (elapsed 52.47 s, remaining 105.57 s) 2023-08-25T04:46:15.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:46:15.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:46:15.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:46:15.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:46:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:46:15.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:46:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:46:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:46:15.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:46:15.236 INFO:teuthology.orchestra.run.smithi145.stderr:16700000 of 50000000 tuples (33%) done (elapsed 52.71 s, remaining 105.10 s) 2023-08-25T04:46:15.426 INFO:teuthology.orchestra.run.smithi145.stderr:16800000 of 50000000 tuples (33%) done (elapsed 52.90 s, remaining 104.53 s) 2023-08-25T04:46:15.604 INFO:teuthology.orchestra.run.smithi145.stderr:16900000 of 50000000 tuples (33%) done (elapsed 53.07 s, remaining 103.95 s) 2023-08-25T04:46:15.819 INFO:teuthology.orchestra.run.smithi145.stderr:17000000 of 50000000 tuples (34%) done (elapsed 53.29 s, remaining 103.44 s) 2023-08-25T04:46:16.068 INFO:teuthology.orchestra.run.smithi145.stderr:17100000 of 50000000 tuples (34%) done (elapsed 53.54 s, remaining 103.01 s) 2023-08-25T04:46:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:15 smithi162 ceph-mon[130941]: pgmap v38: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:15 smithi183 ceph-mon[131414]: pgmap v38: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:16.218 INFO:teuthology.orchestra.run.smithi145.stderr:17200000 of 50000000 tuples (34%) done (elapsed 53.69 s, remaining 102.38 s) 2023-08-25T04:46:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:15 smithi145 ceph-mon[97681]: pgmap v38: 129 pgs: 129 active+clean; 2.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2023-08-25T04:46:16.420 INFO:teuthology.orchestra.run.smithi145.stderr:17300000 of 50000000 tuples (34%) done (elapsed 53.89 s, remaining 101.86 s) 2023-08-25T04:46:16.617 INFO:teuthology.orchestra.run.smithi145.stderr:17400000 of 50000000 tuples (34%) done (elapsed 54.09 s, remaining 101.33 s) 2023-08-25T04:46:16.995 INFO:teuthology.orchestra.run.smithi145.stderr:17500000 of 50000000 tuples (35%) done (elapsed 54.46 s, remaining 101.15 s) 2023-08-25T04:46:18.156 INFO:teuthology.orchestra.run.smithi145.stderr:17600000 of 50000000 tuples (35%) done (elapsed 55.63 s, remaining 102.40 s) 2023-08-25T04:46:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:17 smithi162 ceph-mon[130941]: pgmap v39: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 371 op/s 2023-08-25T04:46:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:17 smithi183 ceph-mon[131414]: pgmap v39: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 371 op/s 2023-08-25T04:46:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:17 smithi145 ceph-mon[97681]: pgmap v39: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 371 op/s 2023-08-25T04:46:18.456 INFO:teuthology.orchestra.run.smithi145.stderr:17700000 of 50000000 tuples (35%) done (elapsed 55.93 s, remaining 102.06 s) 2023-08-25T04:46:18.679 INFO:teuthology.orchestra.run.smithi145.stderr:17800000 of 50000000 tuples (35%) done (elapsed 56.15 s, remaining 101.57 s) 2023-08-25T04:46:18.806 INFO:teuthology.orchestra.run.smithi145.stderr:17900000 of 50000000 tuples (35%) done (elapsed 56.28 s, remaining 100.92 s) 2023-08-25T04:46:19.039 INFO:teuthology.orchestra.run.smithi145.stderr:18000000 of 50000000 tuples (36%) done (elapsed 56.51 s, remaining 100.46 s) 2023-08-25T04:46:19.250 INFO:teuthology.orchestra.run.smithi145.stderr:18100000 of 50000000 tuples (36%) done (elapsed 56.72 s, remaining 99.96 s) 2023-08-25T04:46:19.450 INFO:teuthology.orchestra.run.smithi145.stderr:18200000 of 50000000 tuples (36%) done (elapsed 56.92 s, remaining 99.45 s) 2023-08-25T04:46:19.664 INFO:teuthology.orchestra.run.smithi145.stderr:18300000 of 50000000 tuples (36%) done (elapsed 57.13 s, remaining 98.97 s) 2023-08-25T04:46:19.871 INFO:teuthology.orchestra.run.smithi145.stderr:18400000 of 50000000 tuples (36%) done (elapsed 57.34 s, remaining 98.48 s) 2023-08-25T04:46:20.024 INFO:teuthology.orchestra.run.smithi145.stderr:18500000 of 50000000 tuples (37%) done (elapsed 57.49 s, remaining 97.90 s) 2023-08-25T04:46:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:19 smithi162 ceph-mon[130941]: pgmap v40: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 MiB/s wr, 218 op/s 2023-08-25T04:46:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:19 smithi183 ceph-mon[131414]: pgmap v40: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 MiB/s wr, 218 op/s 2023-08-25T04:46:20.252 INFO:teuthology.orchestra.run.smithi145.stderr:18600000 of 50000000 tuples (37%) done (elapsed 57.72 s, remaining 97.44 s) 2023-08-25T04:46:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:19 smithi145 ceph-mon[97681]: pgmap v40: 129 pgs: 129 active+clean; 3.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 MiB/s wr, 218 op/s 2023-08-25T04:46:20.495 INFO:teuthology.orchestra.run.smithi145.stderr:18700000 of 50000000 tuples (37%) done (elapsed 57.96 s, remaining 97.02 s) 2023-08-25T04:46:20.746 INFO:teuthology.orchestra.run.smithi145.stderr:18800000 of 50000000 tuples (37%) done (elapsed 58.22 s, remaining 96.61 s) 2023-08-25T04:46:20.980 INFO:teuthology.orchestra.run.smithi145.stderr:18900000 of 50000000 tuples (37%) done (elapsed 58.45 s, remaining 96.18 s) 2023-08-25T04:46:21.183 INFO:teuthology.orchestra.run.smithi145.stderr:19000000 of 50000000 tuples (38%) done (elapsed 58.65 s, remaining 95.70 s) 2023-08-25T04:46:21.394 INFO:teuthology.orchestra.run.smithi145.stderr:19100000 of 50000000 tuples (38%) done (elapsed 58.86 s, remaining 95.23 s) 2023-08-25T04:46:21.578 INFO:teuthology.orchestra.run.smithi145.stderr:19200000 of 50000000 tuples (38%) done (elapsed 59.05 s, remaining 94.72 s) 2023-08-25T04:46:21.819 INFO:teuthology.orchestra.run.smithi145.stderr:19300000 of 50000000 tuples (38%) done (elapsed 59.29 s, remaining 94.31 s) 2023-08-25T04:46:21.965 INFO:teuthology.orchestra.run.smithi145.stderr:19400000 of 50000000 tuples (38%) done (elapsed 59.43 s, remaining 93.75 s) 2023-08-25T04:46:22.153 INFO:teuthology.orchestra.run.smithi145.stderr:19500000 of 50000000 tuples (39%) done (elapsed 59.62 s, remaining 93.26 s) 2023-08-25T04:46:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:21 smithi162 ceph-mon[130941]: pgmap v41: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 69 MiB/s wr, 261 op/s 2023-08-25T04:46:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:21 smithi183 ceph-mon[131414]: pgmap v41: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 69 MiB/s wr, 261 op/s 2023-08-25T04:46:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:21 smithi145 ceph-mon[97681]: pgmap v41: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 69 MiB/s wr, 261 op/s 2023-08-25T04:46:22.355 INFO:teuthology.orchestra.run.smithi145.stderr:19600000 of 50000000 tuples (39%) done (elapsed 59.83 s, remaining 92.79 s) 2023-08-25T04:46:22.566 INFO:teuthology.orchestra.run.smithi145.stderr:19700000 of 50000000 tuples (39%) done (elapsed 60.04 s, remaining 92.34 s) 2023-08-25T04:46:22.780 INFO:teuthology.orchestra.run.smithi145.stderr:19800000 of 50000000 tuples (39%) done (elapsed 60.25 s, remaining 91.90 s) 2023-08-25T04:46:22.974 INFO:teuthology.orchestra.run.smithi145.stderr:19900000 of 50000000 tuples (39%) done (elapsed 60.44 s, remaining 91.43 s) 2023-08-25T04:46:23.153 INFO:teuthology.orchestra.run.smithi145.stderr:20000000 of 50000000 tuples (40%) done (elapsed 60.62 s, remaining 90.93 s) 2023-08-25T04:46:23.316 INFO:teuthology.orchestra.run.smithi145.stderr:20100000 of 50000000 tuples (40%) done (elapsed 60.79 s, remaining 90.42 s) 2023-08-25T04:46:23.529 INFO:teuthology.orchestra.run.smithi145.stderr:20200000 of 50000000 tuples (40%) done (elapsed 61.00 s, remaining 89.99 s) 2023-08-25T04:46:23.737 INFO:teuthology.orchestra.run.smithi145.stderr:20300000 of 50000000 tuples (40%) done (elapsed 61.21 s, remaining 89.55 s) 2023-08-25T04:46:23.962 INFO:teuthology.orchestra.run.smithi145.stderr:20400000 of 50000000 tuples (40%) done (elapsed 61.43 s, remaining 89.14 s) 2023-08-25T04:46:24.124 INFO:teuthology.orchestra.run.smithi145.stderr:20500000 of 50000000 tuples (41%) done (elapsed 61.59 s, remaining 88.64 s) 2023-08-25T04:46:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:23 smithi162 ceph-mon[130941]: pgmap v42: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:23 smithi183 ceph-mon[131414]: pgmap v42: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:24.299 INFO:teuthology.orchestra.run.smithi145.stderr:20600000 of 50000000 tuples (41%) done (elapsed 61.77 s, remaining 88.16 s) 2023-08-25T04:46:24.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:23 smithi145 ceph-mon[97681]: pgmap v42: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:24.527 INFO:teuthology.orchestra.run.smithi145.stderr:20700000 of 50000000 tuples (41%) done (elapsed 62.00 s, remaining 87.75 s) 2023-08-25T04:46:24.738 INFO:teuthology.orchestra.run.smithi145.stderr:20800000 of 50000000 tuples (41%) done (elapsed 62.21 s, remaining 87.33 s) 2023-08-25T04:46:24.952 INFO:teuthology.orchestra.run.smithi145.stderr:20900000 of 50000000 tuples (41%) done (elapsed 62.42 s, remaining 86.91 s) 2023-08-25T04:46:25.183 INFO:teuthology.orchestra.run.smithi145.stderr:21000000 of 50000000 tuples (42%) done (elapsed 62.65 s, remaining 86.52 s) 2023-08-25T04:46:25.360 INFO:teuthology.orchestra.run.smithi145.stderr:21100000 of 50000000 tuples (42%) done (elapsed 62.83 s, remaining 86.06 s) 2023-08-25T04:46:25.610 INFO:teuthology.orchestra.run.smithi145.stderr:21200000 of 50000000 tuples (42%) done (elapsed 63.08 s, remaining 85.69 s) 2023-08-25T04:46:25.750 INFO:teuthology.orchestra.run.smithi145.stderr:21300000 of 50000000 tuples (42%) done (elapsed 63.22 s, remaining 85.18 s) 2023-08-25T04:46:25.979 INFO:teuthology.orchestra.run.smithi145.stderr:21400000 of 50000000 tuples (42%) done (elapsed 63.45 s, remaining 84.80 s) 2023-08-25T04:46:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:25 smithi162 ceph-mon[130941]: pgmap v43: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:25 smithi183 ceph-mon[131414]: pgmap v43: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:26.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:25 smithi145 ceph-mon[97681]: pgmap v43: 129 pgs: 129 active+clean; 3.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 68 MiB/s wr, 114 op/s 2023-08-25T04:46:26.566 INFO:teuthology.orchestra.run.smithi145.stderr:21500000 of 50000000 tuples (43%) done (elapsed 64.04 s, remaining 84.89 s) 2023-08-25T04:46:27.083 INFO:teuthology.orchestra.run.smithi145.stderr:21600000 of 50000000 tuples (43%) done (elapsed 64.55 s, remaining 84.87 s) 2023-08-25T04:46:27.428 INFO:teuthology.orchestra.run.smithi145.stderr:21700000 of 50000000 tuples (43%) done (elapsed 64.90 s, remaining 84.64 s) 2023-08-25T04:46:27.638 INFO:teuthology.orchestra.run.smithi145.stderr:21800000 of 50000000 tuples (43%) done (elapsed 65.11 s, remaining 84.22 s) 2023-08-25T04:46:27.903 INFO:teuthology.orchestra.run.smithi145.stderr:21900000 of 50000000 tuples (43%) done (elapsed 65.37 s, remaining 83.88 s) 2023-08-25T04:46:28.124 INFO:teuthology.orchestra.run.smithi145.stderr:22000000 of 50000000 tuples (44%) done (elapsed 65.59 s, remaining 83.48 s) 2023-08-25T04:46:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:27 smithi162 ceph-mon[130941]: pgmap v44: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 89 MiB/s wr, 125 op/s 2023-08-25T04:46:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:27 smithi183 ceph-mon[131414]: pgmap v44: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 89 MiB/s wr, 125 op/s 2023-08-25T04:46:28.263 INFO:teuthology.orchestra.run.smithi145.stderr:22100000 of 50000000 tuples (44%) done (elapsed 65.73 s, remaining 82.98 s) 2023-08-25T04:46:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:27 smithi145 ceph-mon[97681]: pgmap v44: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 89 MiB/s wr, 125 op/s 2023-08-25T04:46:28.455 INFO:teuthology.orchestra.run.smithi145.stderr:22200000 of 50000000 tuples (44%) done (elapsed 65.93 s, remaining 82.55 s) 2023-08-25T04:46:28.687 INFO:teuthology.orchestra.run.smithi145.stderr:22300000 of 50000000 tuples (44%) done (elapsed 66.16 s, remaining 82.18 s) 2023-08-25T04:46:28.901 INFO:teuthology.orchestra.run.smithi145.stderr:22400000 of 50000000 tuples (44%) done (elapsed 66.37 s, remaining 81.78 s) 2023-08-25T04:46:29.086 INFO:teuthology.orchestra.run.smithi145.stderr:22500000 of 50000000 tuples (45%) done (elapsed 66.56 s, remaining 81.35 s) 2023-08-25T04:46:29.277 INFO:teuthology.orchestra.run.smithi145.stderr:22600000 of 50000000 tuples (45%) done (elapsed 66.75 s, remaining 80.92 s) 2023-08-25T04:46:29.432 INFO:teuthology.orchestra.run.smithi145.stderr:22700000 of 50000000 tuples (45%) done (elapsed 66.90 s, remaining 80.46 s) 2023-08-25T04:46:29.728 INFO:teuthology.orchestra.run.smithi145.stderr:22800000 of 50000000 tuples (45%) done (elapsed 67.20 s, remaining 80.17 s) 2023-08-25T04:46:29.838 INFO:teuthology.orchestra.run.smithi145.stderr:22900000 of 50000000 tuples (45%) done (elapsed 67.31 s, remaining 79.65 s) 2023-08-25T04:46:30.059 INFO:teuthology.orchestra.run.smithi145.stderr:23000000 of 50000000 tuples (46%) done (elapsed 67.53 s, remaining 79.27 s) 2023-08-25T04:46:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:29 smithi162 ceph-mon[130941]: pgmap v45: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 62 MiB/s wr, 53 op/s 2023-08-25T04:46:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:29 smithi183 ceph-mon[131414]: pgmap v45: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 62 MiB/s wr, 53 op/s 2023-08-25T04:46:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:23100000 of 50000000 tuples (46%) done (elapsed 67.75 s, remaining 78.90 s) 2023-08-25T04:46:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:29 smithi145 ceph-mon[97681]: pgmap v45: 129 pgs: 129 active+clean; 3.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 62 MiB/s wr, 53 op/s 2023-08-25T04:46:30.501 INFO:teuthology.orchestra.run.smithi145.stderr:23200000 of 50000000 tuples (46%) done (elapsed 67.97 s, remaining 78.52 s) 2023-08-25T04:46:30.945 INFO:teuthology.orchestra.run.smithi145.stderr:23300000 of 50000000 tuples (46%) done (elapsed 68.41 s, remaining 78.40 s) 2023-08-25T04:46:31.424 INFO:teuthology.orchestra.run.smithi145.stderr:23400000 of 50000000 tuples (46%) done (elapsed 68.89 s, remaining 78.31 s) 2023-08-25T04:46:31.585 INFO:teuthology.orchestra.run.smithi145.stderr:23500000 of 50000000 tuples (47%) done (elapsed 69.05 s, remaining 77.87 s) 2023-08-25T04:46:31.810 INFO:teuthology.orchestra.run.smithi145.stderr:23600000 of 50000000 tuples (47%) done (elapsed 69.28 s, remaining 77.50 s) 2023-08-25T04:46:31.998 INFO:teuthology.orchestra.run.smithi145.stderr:23700000 of 50000000 tuples (47%) done (elapsed 69.47 s, remaining 77.09 s) 2023-08-25T04:46:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:31 smithi162 ceph-mon[130941]: pgmap v46: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 122 MiB/s wr, 84 op/s 2023-08-25T04:46:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:31 smithi183 ceph-mon[131414]: pgmap v46: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 122 MiB/s wr, 84 op/s 2023-08-25T04:46:32.206 INFO:teuthology.orchestra.run.smithi145.stderr:23800000 of 50000000 tuples (47%) done (elapsed 69.68 s, remaining 76.70 s) 2023-08-25T04:46:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:31 smithi145 ceph-mon[97681]: pgmap v46: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 122 MiB/s wr, 84 op/s 2023-08-25T04:46:32.407 INFO:teuthology.orchestra.run.smithi145.stderr:23900000 of 50000000 tuples (47%) done (elapsed 69.88 s, remaining 76.31 s) 2023-08-25T04:46:32.610 INFO:teuthology.orchestra.run.smithi145.stderr:24000000 of 50000000 tuples (48%) done (elapsed 70.08 s, remaining 75.92 s) 2023-08-25T04:46:32.760 INFO:teuthology.orchestra.run.smithi145.stderr:24100000 of 50000000 tuples (48%) done (elapsed 70.23 s, remaining 75.48 s) 2023-08-25T04:46:32.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:46:32.978 INFO:teuthology.orchestra.run.smithi145.stderr:24200000 of 50000000 tuples (48%) done (elapsed 70.45 s, remaining 75.11 s) 2023-08-25T04:46:33.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 -- 172.21.15.145:0/3251222787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 msgr2=0x7f87cc106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 --2- 172.21.15.145:0/3251222787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc106b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00faf0 tx=0x7f87bc00fe00 comp rx=0 tx=0).stop 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 -- 172.21.15.145:0/3251222787 shutdown_connections 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 --2- 172.21.15.145:0/3251222787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc1010d0 0x7f87cc10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 --2- 172.21.15.145:0/3251222787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc106b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 --2- 172.21.15.145:0/3251222787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc100710 0x7f87cc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 -- 172.21.15.145:0/3251222787 >> 172.21.15.145:0/3251222787 conn(0x7f87cc0fc1e0 msgr2=0x7f87cc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.030+0000 7f87d119d700 1 -- 172.21.15.145:0/3251222787 shutdown_connections 2023-08-25T04:46:33.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.031+0000 7f87d119d700 1 -- 172.21.15.145:0/3251222787 wait complete. 2023-08-25T04:46:33.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 Processor -- start 2023-08-25T04:46:33.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 -- start start 2023-08-25T04:46:33.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc100710 0x7f87cc194500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 0x7f87cc194a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1994e0 con 0x7f87cc1010d0 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc199650 con 0x7f87cc100710 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d119d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc199810 con 0x7f87cc106720 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.032+0000 7f87d099c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48524/0 (socket says 172.21.15.145:48524) 2023-08-25T04:46:33.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 -- 172.21.15.145:0/402245899 learned_addr learned my addr 172.21.15.145:0/402245899 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87cbfff700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc100710 0x7f87cc194500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 0x7f87cc194a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 -- 172.21.15.145:0/402245899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc100710 msgr2=0x7f87cc194500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc100710 0x7f87cc194500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 -- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 msgr2=0x7f87cc194a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 0x7f87cc194a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 -- 172.21.15.145:0/402245899 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc00f750 con 0x7f87cc106720 2023-08-25T04:46:33.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 0x7f87cc194a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:33.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d099c700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef30 tx=0x7f87c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87c97fa700 1 -- 172.21.15.145:0/402245899 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4018070 con 0x7f87cc106720 2023-08-25T04:46:33.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87c97fa700 1 -- 172.21.15.145:0/402245899 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c400f040 con 0x7f87cc106720 2023-08-25T04:46:33.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc199a90 con 0x7f87cc106720 2023-08-25T04:46:33.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.033+0000 7f87c97fa700 1 -- 172.21.15.145:0/402245899 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400cdb0 con 0x7f87cc106720 2023-08-25T04:46:33.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.034+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc1a1330 con 0x7f87cc106720 2023-08-25T04:46:33.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.038+0000 7f87c97fa700 1 -- 172.21.15.145:0/402245899 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) v1 ==== 134905+0+0 (secure 0 0 0) 0x7f87c4003a00 con 0x7f87cc106720 2023-08-25T04:46:33.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.039+0000 7f87c97fa700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 0x7f87b40a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.039+0000 7f87e5d03b80 1 Processor -- start 2023-08-25T04:46:33.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.039+0000 7f87c97fa700 1 -- 172.21.15.145:0/402245899 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f87c40cb770 con 0x7f87cc106720 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87cbfff700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 0x7f87b40a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 -- start start 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b0a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360a727b0 con 0x561360b10500 2023-08-25T04:46:33.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360a63780 con 0x561360b10e20 2023-08-25T04:46:33.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.040+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360a610a0 con 0x561360b0ac40 2023-08-25T04:46:33.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87d099c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b0a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87d099c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48530/0 (socket says 172.21.15.145:48530) 2023-08-25T04:46:33.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87d099c700 1 -- 172.21.15.145:0/2832265220 learned_addr learned my addr 172.21.15.145:0/2832265220 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 -- 172.21.15.145:0/2832265220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 msgr2=0x561360b0b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cbfff700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 -- 172.21.15.145:0/2832265220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 msgr2=0x561360b108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 -- 172.21.15.145:0/2832265220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613608dc630 con 0x561360b10e20 2023-08-25T04:46:33.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87d099c700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cbfff700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cbfff700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 0x7f87b40a27c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc00ff90 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.041+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b0a700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009ca0 tx=0x7f87c0010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262a86dacb4fc40 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.042+0000 7f87b27fc700 1 -- 172.21.15.145:0/2832265220 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0010bf0 con 0x561360b10e20 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.042+0000 7f87b27fc700 1 -- 172.21.15.145:0/2832265220 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c00044f0 con 0x561360b10e20 2023-08-25T04:46:33.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.042+0000 7f87b27fc700 1 -- 172.21.15.145:0/2832265220 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0012450 con 0x561360b10e20 2023-08-25T04:46:33.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 -- 172.21.15.145:0/2832265220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 msgr2=0x561360b0a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b0a700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009ca0 tx=0x7f87c0010370 comp rx=0 tx=0).stop 2023-08-25T04:46:33.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 -- 172.21.15.145:0/2832265220 shutdown_connections 2023-08-25T04:46:33.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/2832265220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b0a700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 -- 172.21.15.145:0/2832265220 >> 172.21.15.145:0/2832265220 conn(0x561360a7f0e0 msgr2=0x561360b05940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 -- 172.21.15.145:0/2832265220 shutdown_connections 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.043+0000 7f87e5d03b80 1 -- 172.21.15.145:0/2832265220 wait complete. 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.044+0000 7f87e5d03b80 1 -- start start 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.044+0000 7f87e5d03b80 1 Processor -- start 2023-08-25T04:46:33.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b14bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360a5c990 con 0x561360b10500 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613609fabb0 con 0x561360b10e20 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613609fa8b0 con 0x561360b0ac40 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87e5d03b80 10 client.0 _ll_register_callbacks cb 0x561360944a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48540/0 (socket says 172.21.15.145:48540) 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 -- 172.21.15.145:0/1789005711 learned_addr learned my addr 172.21.15.145:0/1789005711 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:33.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87d099c700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b14bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 -- 172.21.15.145:0/1789005711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 msgr2=0x561360b14bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b14bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 -- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 msgr2=0x561360b146b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b146b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cbfff700 1 -- 172.21.15.145:0/1789005711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc1a1330 con 0x561360b0ac40 2023-08-25T04:46:33.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.045+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87cbfff700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87bc018000 tx=0x7f87bc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cee84aac2d5efe71 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc005270 con 0x561360b0ac40 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f87bc00f440 con 0x561360b0ac40 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561360ada1c0 con 0x561360b0ac40 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87bc0053d0 con 0x561360b0ac40 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87e5d03b80 10 client.25804 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:33.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.046+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f87c0009900 con 0x561360b0ac40 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561360ada1c0 con 0x561360b0ac40 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f87bc034020 con 0x561360b0ac40 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 20 client.25804 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b24a00 0x561360b26e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 4 client.25804 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:46:33.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561360a02ba0 con 0x561360b24a00 2023-08-25T04:46:33.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.047+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b24a00 0x561360b26e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.048+0000 7f87ab7fe700 10 client.25804 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:33.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.048+0000 7f87cb7fe700 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b24a00 0x561360b26e20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373a17d430fd5ce0 server_cookie=72b2c330a18ffb6a in_seq=0 out_seq=0 2023-08-25T04:46:33.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.048+0000 7f87ab7fe700 1 -- 172.21.15.145:0/1789005711 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561360a02ba0 con 0x561360b24a00 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.048+0000 7f87ab7fe700 10 client.25804 handle_command_reply: tid=0 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 client.25804 shutdown 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 msgr2=0x561360b14170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87bc018000 tx=0x7f87bc009310 comp rx=0 tx=0).stop 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 shutdown_connections 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10e20 0x561360b14bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b24a00 0x561360b26e20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b14170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87ab7fe700 0 client.25804 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/1789005711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10500 0x561360b146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 >> 172.21.15.145:0/1789005711 conn(0x561360a7f0e0 msgr2=0x561360b096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 shutdown_connections 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 1 -- 172.21.15.145:0/1789005711 wait complete. 2023-08-25T04:46:33.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.049+0000 7f87e5d03b80 20 client.25804 trim_cache size 0 max 16384 2023-08-25T04:46:33.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 Processor -- start 2023-08-25T04:46:33.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 -- start start 2023-08-25T04:46:33.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613608de920 con 0x561360b10e20 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360b01a00 con 0x561360b10500 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.090+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360b0ebc0 con 0x561360b0ac40 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50398/0 (socket says 172.21.15.145:50398) 2023-08-25T04:46:33.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 -- 172.21.15.145:0/815339945 learned_addr learned my addr 172.21.15.145:0/815339945 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 -- 172.21.15.145:0/815339945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 msgr2=0x561360b0a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 -- 172.21.15.145:0/815339945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 msgr2=0x561360b0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.091+0000 7f87d099c700 1 -- 172.21.15.145:0/815339945 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561360b2ba30 con 0x561360b10e20 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.092+0000 7f87d099c700 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef00 tx=0x7f87c40985f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d92dc1d14c036 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.092+0000 7f87b2ffd700 1 -- 172.21.15.145:0/815339945 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c401b320 con 0x561360b10e20 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.092+0000 7f87b2ffd700 1 -- 172.21.15.145:0/815339945 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c4099080 con 0x561360b10e20 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.092+0000 7f87b2ffd700 1 -- 172.21.15.145:0/815339945 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4098be0 con 0x561360b10e20 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 -- 172.21.15.145:0/815339945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 msgr2=0x561360b14a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef00 tx=0x7f87c40985f0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 -- 172.21.15.145:0/815339945 shutdown_connections 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b14a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b0a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/815339945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 -- 172.21.15.145:0/815339945 >> 172.21.15.145:0/815339945 conn(0x561360a7f0e0 msgr2=0x561360b0df70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 -- 172.21.15.145:0/815339945 shutdown_connections 2023-08-25T04:46:33.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.093+0000 7f87e5d03b80 1 -- 172.21.15.145:0/815339945 wait complete. 2023-08-25T04:46:33.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.094+0000 7f87e5d03b80 1 -- start start 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.094+0000 7f87e5d03b80 1 Processor -- start 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b1a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360b77d80 con 0x561360b10e20 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360b77ef0 con 0x561360b10500 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561360b780b0 con 0x561360b0ac40 2023-08-25T04:46:33.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87e5d03b80 10 client.0 _ll_register_callbacks cb 0x561360944a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.095+0000 7f87d099c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50410/0 (socket says 172.21.15.145:50410) 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 -- 172.21.15.145:0/610212742 learned_addr learned my addr 172.21.15.145:0/610212742 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87cbfff700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 -- 172.21.15.145:0/610212742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 msgr2=0x561360b19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b19b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 -- 172.21.15.145:0/610212742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 msgr2=0x561360b1a0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b1a0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 -- 172.21.15.145:0/610212742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc199a90 con 0x561360b10e20 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87d099c700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f87c4098680 tx=0x7f87c4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e00cd78f564372 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.096+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c409c7b0 con 0x561360b10e20 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f87c40a0700 con 0x561360b10e20 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561360afd8b0 con 0x561360b10e20 2023-08-25T04:46:33.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c4098be0 con 0x561360b10e20 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 10 client.15786 fetch_fsmap learned FSMap version 27 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561360888de0 con 0x561360b10e20 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c40c8020 con 0x561360b10e20 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f87c40c8e00 con 0x561360b10e20 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.097+0000 7f87e5d03b80 20 client.15786 populate_metadata read hostname 'smithi145' 2023-08-25T04:46:33.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b793a0 0x561360b7b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87e5d03b80 4 client.15786 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561360a55970 con 0x561360b793a0 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87cbfff700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b793a0 0x561360b7b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87cbfff700 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b793a0 0x561360b7b7c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f45b8fe37d7fcc server_cookie=94cd63131d13c0ab in_seq=0 out_seq=0 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.098+0000 7f87ab7fe700 10 client.15786 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.099+0000 7f87ab7fe700 1 -- 172.21.15.145:0/610212742 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561360a55970 con 0x561360b793a0 2023-08-25T04:46:33.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.099+0000 7f87ab7fe700 10 client.15786 handle_command_reply: tid=0 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.099+0000 7f87e5d03b80 1 client.15786 shutdown 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 msgr2=0x561360b77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f87c4098680 tx=0x7f87c4004dc0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 shutdown_connections 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561360b10e20 0x561360b77810 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561360b793a0 0x561360b7b7c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561360b0ac40 0x561360b19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 --2- 172.21.15.145:0/610212742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561360b10500 0x561360b1a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87ab7fe700 0 client.15786 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:46:33.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 >> 172.21.15.145:0/610212742 conn(0x561360a7f0e0 msgr2=0x561360b050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 shutdown_connections 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.100+0000 7f87e5d03b80 1 -- 172.21.15.145:0/610212742 wait complete. 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87e5d03b80 20 client.15786 trim_cache size 0 max 16384 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:46:33.109 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:46:33.110 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:46:33.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 msgr2=0x7f87b40a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 0x7f87b40a27c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc00ff90 comp rx=0 tx=0).stop 2023-08-25T04:46:33.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 msgr2=0x7f87cc198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:46:33.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87c400ef30 tx=0x7f87c400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 shutdown_connections 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87cc106720 0x7f87cc198ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f87b40a0300 0x7f87b40a27c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc100710 0x7f87cc194500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 --2- 172.21.15.145:0/402245899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87cc1010d0 0x7f87cc194a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.101+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 >> 172.21.15.145:0/402245899 conn(0x7f87cc0fc1e0 msgr2=0x7f87cc0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.102+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 shutdown_connections 2023-08-25T04:46:33.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:46:33.102+0000 7f87d119d700 1 -- 172.21.15.145:0/402245899 wait complete. 2023-08-25T04:46:33.123 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:46:33.175 INFO:teuthology.orchestra.run.smithi145.stderr:24300000 of 50000000 tuples (48%) done (elapsed 70.64 s, remaining 74.72 s) 2023-08-25T04:46:33.564 INFO:teuthology.orchestra.run.smithi145.stderr:24400000 of 50000000 tuples (48%) done (elapsed 71.03 s, remaining 74.53 s) 2023-08-25T04:46:34.026 INFO:teuthology.orchestra.run.smithi145.stderr:24500000 of 50000000 tuples (49%) done (elapsed 71.50 s, remaining 74.41 s) 2023-08-25T04:46:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:33 smithi162 ceph-mon[130941]: pgmap v47: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:33 smithi183 ceph-mon[131414]: pgmap v47: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:34.195 INFO:teuthology.orchestra.run.smithi145.stderr:24600000 of 50000000 tuples (49%) done (elapsed 71.66 s, remaining 74.00 s) 2023-08-25T04:46:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:33 smithi145 ceph-mon[97681]: pgmap v47: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:34.344 INFO:teuthology.orchestra.run.smithi145.stderr:24700000 of 50000000 tuples (49%) done (elapsed 71.81 s, remaining 73.56 s) 2023-08-25T04:46:34.563 INFO:teuthology.orchestra.run.smithi145.stderr:24800000 of 50000000 tuples (49%) done (elapsed 72.03 s, remaining 73.19 s) 2023-08-25T04:46:34.773 INFO:teuthology.orchestra.run.smithi145.stderr:24900000 of 50000000 tuples (49%) done (elapsed 72.24 s, remaining 72.82 s) 2023-08-25T04:46:34.973 INFO:teuthology.orchestra.run.smithi145.stderr:25000000 of 50000000 tuples (50%) done (elapsed 72.44 s, remaining 72.44 s) 2023-08-25T04:46:35.192 INFO:teuthology.orchestra.run.smithi145.stderr:25100000 of 50000000 tuples (50%) done (elapsed 72.66 s, remaining 72.08 s) 2023-08-25T04:46:35.352 INFO:teuthology.orchestra.run.smithi145.stderr:25200000 of 50000000 tuples (50%) done (elapsed 72.82 s, remaining 71.67 s) 2023-08-25T04:46:35.605 INFO:teuthology.orchestra.run.smithi145.stderr:25300000 of 50000000 tuples (50%) done (elapsed 73.07 s, remaining 71.34 s) 2023-08-25T04:46:35.832 INFO:teuthology.orchestra.run.smithi145.stderr:25400000 of 50000000 tuples (50%) done (elapsed 73.30 s, remaining 70.99 s) 2023-08-25T04:46:36.184 INFO:teuthology.orchestra.run.smithi145.stderr:25500000 of 50000000 tuples (51%) done (elapsed 73.65 s, remaining 70.77 s) 2023-08-25T04:46:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:35 smithi145 ceph-mon[97681]: pgmap v48: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:35 smithi162 ceph-mon[130941]: pgmap v48: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:35 smithi183 ceph-mon[131414]: pgmap v48: 129 pgs: 129 active+clean; 4.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 81 MiB/s wr, 42 op/s 2023-08-25T04:46:36.673 INFO:teuthology.orchestra.run.smithi145.stderr:25600000 of 50000000 tuples (51%) done (elapsed 74.14 s, remaining 70.67 s) 2023-08-25T04:46:37.513 INFO:teuthology.orchestra.run.smithi145.stderr:25700000 of 50000000 tuples (51%) done (elapsed 74.98 s, remaining 70.90 s) 2023-08-25T04:46:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:25800000 of 50000000 tuples (51%) done (elapsed 75.20 s, remaining 70.53 s) 2023-08-25T04:46:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:25900000 of 50000000 tuples (51%) done (elapsed 75.42 s, remaining 70.18 s) 2023-08-25T04:46:38.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:37 smithi145 ceph-mon[97681]: pgmap v49: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 99 MiB/s wr, 96 op/s 2023-08-25T04:46:38.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:37 smithi145 ceph-mon[97681]: mgrmap e24: x(active, since 92s), standbys: y, z 2023-08-25T04:46:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:37 smithi162 ceph-mon[130941]: pgmap v49: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 99 MiB/s wr, 96 op/s 2023-08-25T04:46:38.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:37 smithi162 ceph-mon[130941]: mgrmap e24: x(active, since 92s), standbys: y, z 2023-08-25T04:46:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:37 smithi183 ceph-mon[131414]: pgmap v49: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 99 MiB/s wr, 96 op/s 2023-08-25T04:46:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:37 smithi183 ceph-mon[131414]: mgrmap e24: x(active, since 92s), standbys: y, z 2023-08-25T04:46:38.626 INFO:teuthology.orchestra.run.smithi145.stderr:26000000 of 50000000 tuples (52%) done (elapsed 76.10 s, remaining 70.24 s) 2023-08-25T04:46:38.933 INFO:teuthology.orchestra.run.smithi145.stderr:26100000 of 50000000 tuples (52%) done (elapsed 76.40 s, remaining 69.96 s) 2023-08-25T04:46:39.144 INFO:teuthology.orchestra.run.smithi145.stderr:26200000 of 50000000 tuples (52%) done (elapsed 76.61 s, remaining 69.60 s) 2023-08-25T04:46:39.347 INFO:teuthology.orchestra.run.smithi145.stderr:26300000 of 50000000 tuples (52%) done (elapsed 76.82 s, remaining 69.22 s) 2023-08-25T04:46:39.585 INFO:teuthology.orchestra.run.smithi145.stderr:26400000 of 50000000 tuples (52%) done (elapsed 77.05 s, remaining 68.88 s) 2023-08-25T04:46:39.719 INFO:teuthology.orchestra.run.smithi145.stderr:26500000 of 50000000 tuples (53%) done (elapsed 77.19 s, remaining 68.45 s) 2023-08-25T04:46:39.940 INFO:teuthology.orchestra.run.smithi145.stderr:26600000 of 50000000 tuples (53%) done (elapsed 77.41 s, remaining 68.10 s) 2023-08-25T04:46:40.159 INFO:teuthology.orchestra.run.smithi145.stderr:26700000 of 50000000 tuples (53%) done (elapsed 77.63 s, remaining 67.74 s) 2023-08-25T04:46:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:39 smithi145 ceph-mon[97681]: pgmap v50: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 MiB/s wr, 85 op/s 2023-08-25T04:46:40.338 INFO:teuthology.orchestra.run.smithi145.stderr:26800000 of 50000000 tuples (53%) done (elapsed 77.81 s, remaining 67.36 s) 2023-08-25T04:46:40.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:39 smithi162 ceph-mon[130941]: pgmap v50: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 MiB/s wr, 85 op/s 2023-08-25T04:46:40.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:39 smithi183 ceph-mon[131414]: pgmap v50: 129 pgs: 129 active+clean; 4.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 MiB/s wr, 85 op/s 2023-08-25T04:46:40.565 INFO:teuthology.orchestra.run.smithi145.stderr:26900000 of 50000000 tuples (53%) done (elapsed 78.03 s, remaining 67.01 s) 2023-08-25T04:46:40.788 INFO:teuthology.orchestra.run.smithi145.stderr:27000000 of 50000000 tuples (54%) done (elapsed 78.26 s, remaining 66.66 s) 2023-08-25T04:46:40.934 INFO:teuthology.orchestra.run.smithi145.stderr:27100000 of 50000000 tuples (54%) done (elapsed 78.40 s, remaining 66.25 s) 2023-08-25T04:46:41.163 INFO:teuthology.orchestra.run.smithi145.stderr:27200000 of 50000000 tuples (54%) done (elapsed 78.63 s, remaining 65.91 s) 2023-08-25T04:46:41.391 INFO:teuthology.orchestra.run.smithi145.stderr:27300000 of 50000000 tuples (54%) done (elapsed 78.86 s, remaining 65.57 s) 2023-08-25T04:46:41.646 INFO:teuthology.orchestra.run.smithi145.stderr:27400000 of 50000000 tuples (54%) done (elapsed 79.11 s, remaining 65.25 s) 2023-08-25T04:46:41.809 INFO:teuthology.orchestra.run.smithi145.stderr:27500000 of 50000000 tuples (55%) done (elapsed 79.28 s, remaining 64.86 s) 2023-08-25T04:46:41.986 INFO:teuthology.orchestra.run.smithi145.stderr:27600000 of 50000000 tuples (55%) done (elapsed 79.46 s, remaining 64.49 s) 2023-08-25T04:46:42.121 INFO:teuthology.orchestra.run.smithi145.stderr:27700000 of 50000000 tuples (55%) done (elapsed 79.59 s, remaining 64.07 s) 2023-08-25T04:46:42.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:41 smithi145 ceph-mon[97681]: pgmap v51: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 137 MiB/s wr, 141 op/s 2023-08-25T04:46:42.342 INFO:teuthology.orchestra.run.smithi145.stderr:27800000 of 50000000 tuples (55%) done (elapsed 79.81 s, remaining 63.73 s) 2023-08-25T04:46:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:41 smithi162 ceph-mon[130941]: pgmap v51: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 137 MiB/s wr, 141 op/s 2023-08-25T04:46:42.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:41 smithi183 ceph-mon[131414]: pgmap v51: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 137 MiB/s wr, 141 op/s 2023-08-25T04:46:42.554 INFO:teuthology.orchestra.run.smithi145.stderr:27900000 of 50000000 tuples (55%) done (elapsed 80.02 s, remaining 63.39 s) 2023-08-25T04:46:42.765 INFO:teuthology.orchestra.run.smithi145.stderr:28000000 of 50000000 tuples (56%) done (elapsed 80.23 s, remaining 63.04 s) 2023-08-25T04:46:42.970 INFO:teuthology.orchestra.run.smithi145.stderr:28100000 of 50000000 tuples (56%) done (elapsed 80.44 s, remaining 62.69 s) 2023-08-25T04:46:43.172 INFO:teuthology.orchestra.run.smithi145.stderr:28200000 of 50000000 tuples (56%) done (elapsed 80.64 s, remaining 62.34 s) 2023-08-25T04:46:43.338 INFO:teuthology.orchestra.run.smithi145.stderr:28300000 of 50000000 tuples (56%) done (elapsed 80.81 s, remaining 61.96 s) 2023-08-25T04:46:43.550 INFO:teuthology.orchestra.run.smithi145.stderr:28400000 of 50000000 tuples (56%) done (elapsed 81.02 s, remaining 61.62 s) 2023-08-25T04:46:43.773 INFO:teuthology.orchestra.run.smithi145.stderr:28500000 of 50000000 tuples (57%) done (elapsed 81.24 s, remaining 61.29 s) 2023-08-25T04:46:44.005 INFO:teuthology.orchestra.run.smithi145.stderr:28600000 of 50000000 tuples (57%) done (elapsed 81.47 s, remaining 60.96 s) 2023-08-25T04:46:44.173 INFO:teuthology.orchestra.run.smithi145.stderr:28700000 of 50000000 tuples (57%) done (elapsed 81.64 s, remaining 60.59 s) 2023-08-25T04:46:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:43 smithi145 ceph-mon[97681]: pgmap v52: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:43 smithi162 ceph-mon[130941]: pgmap v52: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:44.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:43 smithi183 ceph-mon[131414]: pgmap v52: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:44.424 INFO:teuthology.orchestra.run.smithi145.stderr:28800000 of 50000000 tuples (57%) done (elapsed 81.89 s, remaining 60.28 s) 2023-08-25T04:46:44.656 INFO:teuthology.orchestra.run.smithi145.stderr:28900000 of 50000000 tuples (57%) done (elapsed 82.13 s, remaining 59.96 s) 2023-08-25T04:46:44.788 INFO:teuthology.orchestra.run.smithi145.stderr:29000000 of 50000000 tuples (58%) done (elapsed 82.26 s, remaining 59.57 s) 2023-08-25T04:46:45.001 INFO:teuthology.orchestra.run.smithi145.stderr:29100000 of 50000000 tuples (58%) done (elapsed 82.47 s, remaining 59.23 s) 2023-08-25T04:46:45.211 INFO:teuthology.orchestra.run.smithi145.stderr:29200000 of 50000000 tuples (58%) done (elapsed 82.68 s, remaining 58.90 s) 2023-08-25T04:46:45.392 INFO:teuthology.orchestra.run.smithi145.stderr:29300000 of 50000000 tuples (58%) done (elapsed 82.86 s, remaining 58.54 s) 2023-08-25T04:46:45.801 INFO:teuthology.orchestra.run.smithi145.stderr:29400000 of 50000000 tuples (58%) done (elapsed 83.27 s, remaining 58.35 s) 2023-08-25T04:46:46.343 INFO:teuthology.orchestra.run.smithi145.stderr:29500000 of 50000000 tuples (59%) done (elapsed 83.81 s, remaining 58.24 s) 2023-08-25T04:46:46.355 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:45 smithi145 ceph-mon[97681]: pgmap v53: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:45 smithi162 ceph-mon[130941]: pgmap v53: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:46.427 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:45 smithi183 ceph-mon[131414]: pgmap v53: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 MiB/s wr, 110 op/s 2023-08-25T04:46:47.044 INFO:teuthology.orchestra.run.smithi145.stderr:29600000 of 50000000 tuples (59%) done (elapsed 84.51 s, remaining 58.25 s) 2023-08-25T04:46:47.272 INFO:teuthology.orchestra.run.smithi145.stderr:29700000 of 50000000 tuples (59%) done (elapsed 84.74 s, remaining 57.92 s) 2023-08-25T04:46:47.501 INFO:teuthology.orchestra.run.smithi145.stderr:29800000 of 50000000 tuples (59%) done (elapsed 84.97 s, remaining 57.60 s) 2023-08-25T04:46:47.610 INFO:teuthology.orchestra.run.smithi145.stderr:29900000 of 50000000 tuples (59%) done (elapsed 85.08 s, remaining 57.19 s) 2023-08-25T04:46:47.855 INFO:teuthology.orchestra.run.smithi145.stderr:30000000 of 50000000 tuples (60%) done (elapsed 85.32 s, remaining 56.88 s) 2023-08-25T04:46:48.074 INFO:teuthology.orchestra.run.smithi145.stderr:30100000 of 50000000 tuples (60%) done (elapsed 85.54 s, remaining 56.56 s) 2023-08-25T04:46:48.313 INFO:teuthology.orchestra.run.smithi145.stderr:30200000 of 50000000 tuples (60%) done (elapsed 85.78 s, remaining 56.24 s) 2023-08-25T04:46:48.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:47 smithi145 ceph-mon[97681]: pgmap v54: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 MiB/s wr, 122 op/s 2023-08-25T04:46:48.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:47 smithi162 ceph-mon[130941]: pgmap v54: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 MiB/s wr, 122 op/s 2023-08-25T04:46:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:47 smithi183 ceph-mon[131414]: pgmap v54: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 MiB/s wr, 122 op/s 2023-08-25T04:46:48.528 INFO:teuthology.orchestra.run.smithi145.stderr:30300000 of 50000000 tuples (60%) done (elapsed 86.00 s, remaining 55.91 s) 2023-08-25T04:46:48.752 INFO:teuthology.orchestra.run.smithi145.stderr:30400000 of 50000000 tuples (60%) done (elapsed 86.22 s, remaining 55.59 s) 2023-08-25T04:46:48.909 INFO:teuthology.orchestra.run.smithi145.stderr:30500000 of 50000000 tuples (61%) done (elapsed 86.38 s, remaining 55.23 s) 2023-08-25T04:46:49.097 INFO:teuthology.orchestra.run.smithi145.stderr:30600000 of 50000000 tuples (61%) done (elapsed 86.57 s, remaining 54.88 s) 2023-08-25T04:46:49.327 INFO:teuthology.orchestra.run.smithi145.stderr:30700000 of 50000000 tuples (61%) done (elapsed 86.80 s, remaining 54.57 s) 2023-08-25T04:46:49.524 INFO:teuthology.orchestra.run.smithi145.stderr:30800000 of 50000000 tuples (61%) done (elapsed 86.99 s, remaining 54.23 s) 2023-08-25T04:46:49.741 INFO:teuthology.orchestra.run.smithi145.stderr:30900000 of 50000000 tuples (61%) done (elapsed 87.21 s, remaining 53.91 s) 2023-08-25T04:46:49.949 INFO:teuthology.orchestra.run.smithi145.stderr:31000000 of 50000000 tuples (62%) done (elapsed 87.42 s, remaining 53.58 s) 2023-08-25T04:46:50.080 INFO:teuthology.orchestra.run.smithi145.stderr:31100000 of 50000000 tuples (62%) done (elapsed 87.55 s, remaining 53.21 s) 2023-08-25T04:46:50.273 INFO:teuthology.orchestra.run.smithi145.stderr:31200000 of 50000000 tuples (62%) done (elapsed 87.74 s, remaining 52.87 s) 2023-08-25T04:46:50.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:49 smithi145 ceph-mon[97681]: pgmap v55: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 MiB/s wr, 68 op/s 2023-08-25T04:46:50.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:49 smithi162 ceph-mon[130941]: pgmap v55: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 MiB/s wr, 68 op/s 2023-08-25T04:46:50.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:49 smithi183 ceph-mon[131414]: pgmap v55: 129 pgs: 129 active+clean; 5.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 MiB/s wr, 68 op/s 2023-08-25T04:46:50.494 INFO:teuthology.orchestra.run.smithi145.stderr:31300000 of 50000000 tuples (62%) done (elapsed 87.96 s, remaining 52.55 s) 2023-08-25T04:46:50.711 INFO:teuthology.orchestra.run.smithi145.stderr:31400000 of 50000000 tuples (62%) done (elapsed 88.18 s, remaining 52.23 s) 2023-08-25T04:46:50.821 INFO:teuthology.orchestra.run.smithi145.stderr:31500000 of 50000000 tuples (63%) done (elapsed 88.29 s, remaining 51.85 s) 2023-08-25T04:46:51.014 INFO:teuthology.orchestra.run.smithi145.stderr:31600000 of 50000000 tuples (63%) done (elapsed 88.48 s, remaining 51.52 s) 2023-08-25T04:46:51.208 INFO:teuthology.orchestra.run.smithi145.stderr:31700000 of 50000000 tuples (63%) done (elapsed 88.68 s, remaining 51.19 s) 2023-08-25T04:46:51.394 INFO:teuthology.orchestra.run.smithi145.stderr:31800000 of 50000000 tuples (63%) done (elapsed 88.86 s, remaining 50.86 s) 2023-08-25T04:46:51.561 INFO:teuthology.orchestra.run.smithi145.stderr:31900000 of 50000000 tuples (63%) done (elapsed 89.03 s, remaining 50.52 s) 2023-08-25T04:46:51.739 INFO:teuthology.orchestra.run.smithi145.stderr:32000000 of 50000000 tuples (64%) done (elapsed 89.21 s, remaining 50.18 s) 2023-08-25T04:46:51.948 INFO:teuthology.orchestra.run.smithi145.stderr:32100000 of 50000000 tuples (64%) done (elapsed 89.42 s, remaining 49.86 s) 2023-08-25T04:46:52.169 INFO:teuthology.orchestra.run.smithi145.stderr:32200000 of 50000000 tuples (64%) done (elapsed 89.64 s, remaining 49.55 s) 2023-08-25T04:46:52.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:51 smithi145 ceph-mon[97681]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 139 MiB/s wr, 96 op/s 2023-08-25T04:46:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:32300000 of 50000000 tuples (64%) done (elapsed 89.84 s, remaining 49.23 s) 2023-08-25T04:46:52.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:51 smithi162 ceph-mon[130941]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 139 MiB/s wr, 96 op/s 2023-08-25T04:46:52.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:51 smithi183 ceph-mon[131414]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 139 MiB/s wr, 96 op/s 2023-08-25T04:46:52.588 INFO:teuthology.orchestra.run.smithi145.stderr:32400000 of 50000000 tuples (64%) done (elapsed 90.06 s, remaining 48.92 s) 2023-08-25T04:46:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:32500000 of 50000000 tuples (65%) done (elapsed 90.24 s, remaining 48.59 s) 2023-08-25T04:46:53.016 INFO:teuthology.orchestra.run.smithi145.stderr:32600000 of 50000000 tuples (65%) done (elapsed 90.48 s, remaining 48.30 s) 2023-08-25T04:46:53.139 INFO:teuthology.orchestra.run.smithi145.stderr:32700000 of 50000000 tuples (65%) done (elapsed 90.61 s, remaining 47.94 s) 2023-08-25T04:46:53.351 INFO:teuthology.orchestra.run.smithi145.stderr:32800000 of 50000000 tuples (65%) done (elapsed 90.82 s, remaining 47.63 s) 2023-08-25T04:46:53.538 INFO:teuthology.orchestra.run.smithi145.stderr:32900000 of 50000000 tuples (65%) done (elapsed 91.01 s, remaining 47.30 s) 2023-08-25T04:46:53.755 INFO:teuthology.orchestra.run.smithi145.stderr:33000000 of 50000000 tuples (66%) done (elapsed 91.22 s, remaining 46.99 s) 2023-08-25T04:46:53.978 INFO:teuthology.orchestra.run.smithi145.stderr:33100000 of 50000000 tuples (66%) done (elapsed 91.45 s, remaining 46.69 s) 2023-08-25T04:46:54.307 INFO:teuthology.orchestra.run.smithi145.stderr:33200000 of 50000000 tuples (66%) done (elapsed 91.78 s, remaining 46.44 s) 2023-08-25T04:46:54.338 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:53 smithi145 ceph-mon[97681]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:53 smithi162 ceph-mon[130941]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:54.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:53 smithi183 ceph-mon[131414]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:54.685 INFO:teuthology.orchestra.run.smithi145.stderr:33300000 of 50000000 tuples (66%) done (elapsed 92.15 s, remaining 46.22 s) 2023-08-25T04:46:55.056 INFO:teuthology.orchestra.run.smithi145.stderr:33400000 of 50000000 tuples (66%) done (elapsed 92.52 s, remaining 45.99 s) 2023-08-25T04:46:55.941 INFO:teuthology.orchestra.run.smithi145.stderr:33500000 of 50000000 tuples (67%) done (elapsed 93.41 s, remaining 46.01 s) 2023-08-25T04:46:56.173 INFO:teuthology.orchestra.run.smithi145.stderr:33600000 of 50000000 tuples (67%) done (elapsed 93.64 s, remaining 45.71 s) 2023-08-25T04:46:56.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:55 smithi145 ceph-mon[97681]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:56.378 INFO:teuthology.orchestra.run.smithi145.stderr:33700000 of 50000000 tuples (67%) done (elapsed 93.85 s, remaining 45.39 s) 2023-08-25T04:46:56.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:55 smithi162 ceph-mon[130941]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:56.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:55 smithi183 ceph-mon[131414]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 80 MiB/s wr, 39 op/s 2023-08-25T04:46:56.515 INFO:teuthology.orchestra.run.smithi145.stderr:33800000 of 50000000 tuples (67%) done (elapsed 93.99 s, remaining 45.05 s) 2023-08-25T04:46:56.780 INFO:teuthology.orchestra.run.smithi145.stderr:33900000 of 50000000 tuples (67%) done (elapsed 94.25 s, remaining 44.76 s) 2023-08-25T04:46:56.930 INFO:teuthology.orchestra.run.smithi145.stderr:34000000 of 50000000 tuples (68%) done (elapsed 94.40 s, remaining 44.42 s) 2023-08-25T04:46:57.147 INFO:teuthology.orchestra.run.smithi145.stderr:34100000 of 50000000 tuples (68%) done (elapsed 94.62 s, remaining 44.12 s) 2023-08-25T04:46:57.364 INFO:teuthology.orchestra.run.smithi145.stderr:34200000 of 50000000 tuples (68%) done (elapsed 94.83 s, remaining 43.81 s) 2023-08-25T04:46:57.579 INFO:teuthology.orchestra.run.smithi145.stderr:34300000 of 50000000 tuples (68%) done (elapsed 95.05 s, remaining 43.51 s) 2023-08-25T04:46:57.757 INFO:teuthology.orchestra.run.smithi145.stderr:34400000 of 50000000 tuples (68%) done (elapsed 95.23 s, remaining 43.18 s) 2023-08-25T04:46:57.962 INFO:teuthology.orchestra.run.smithi145.stderr:34500000 of 50000000 tuples (69%) done (elapsed 95.43 s, remaining 42.88 s) 2023-08-25T04:46:58.162 INFO:teuthology.orchestra.run.smithi145.stderr:34600000 of 50000000 tuples (69%) done (elapsed 95.63 s, remaining 42.56 s) 2023-08-25T04:46:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:57 smithi145 ceph-mon[97681]: pgmap v59: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 132 MiB/s wr, 61 op/s 2023-08-25T04:46:58.373 INFO:teuthology.orchestra.run.smithi145.stderr:34700000 of 50000000 tuples (69%) done (elapsed 95.84 s, remaining 42.26 s) 2023-08-25T04:46:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:57 smithi162 ceph-mon[130941]: pgmap v59: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 132 MiB/s wr, 61 op/s 2023-08-25T04:46:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:57 smithi183 ceph-mon[131414]: pgmap v59: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 132 MiB/s wr, 61 op/s 2023-08-25T04:46:58.560 INFO:teuthology.orchestra.run.smithi145.stderr:34800000 of 50000000 tuples (69%) done (elapsed 96.03 s, remaining 41.94 s) 2023-08-25T04:46:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:34900000 of 50000000 tuples (69%) done (elapsed 96.27 s, remaining 41.65 s) 2023-08-25T04:46:59.045 INFO:teuthology.orchestra.run.smithi145.stderr:35000000 of 50000000 tuples (70%) done (elapsed 96.52 s, remaining 41.36 s) 2023-08-25T04:46:59.703 INFO:teuthology.orchestra.run.smithi145.stderr:35100000 of 50000000 tuples (70%) done (elapsed 97.17 s, remaining 41.25 s) 2023-08-25T04:47:00.185 INFO:teuthology.orchestra.run.smithi145.stderr:35200000 of 50000000 tuples (70%) done (elapsed 97.65 s, remaining 41.06 s) 2023-08-25T04:47:00.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:46:59 smithi145 ceph-mon[97681]: pgmap v60: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 110 MiB/s wr, 49 op/s 2023-08-25T04:47:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:46:59 smithi162 ceph-mon[130941]: pgmap v60: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 110 MiB/s wr, 49 op/s 2023-08-25T04:47:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:46:59 smithi183 ceph-mon[131414]: pgmap v60: 129 pgs: 129 active+clean; 6.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 110 MiB/s wr, 49 op/s 2023-08-25T04:47:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:01 smithi145 ceph-mon[97681]: pgmap v61: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 137 MiB/s wr, 130 op/s 2023-08-25T04:47:02.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:01 smithi162 ceph-mon[130941]: pgmap v61: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 137 MiB/s wr, 130 op/s 2023-08-25T04:47:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:01 smithi183 ceph-mon[131414]: pgmap v61: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 137 MiB/s wr, 130 op/s 2023-08-25T04:47:03.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:47:03.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:02 smithi145 ceph-mon[97681]: pgmap v62: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:03.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 -- 172.21.15.145:0/3320095250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 msgr2=0x7f14440ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 --2- 172.21.15.145:0/3320095250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440ffbe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f14380098f0 tx=0x7f1438009c00 comp rx=0 tx=0).stop 2023-08-25T04:47:03.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 -- 172.21.15.145:0/3320095250 shutdown_connections 2023-08-25T04:47:03.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 --2- 172.21.15.145:0/3320095250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444100120 0x7f144410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.391 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 --2- 172.21.15.145:0/3320095250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444105770 0x7f1444105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 --2- 172.21.15.145:0/3320095250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440ffbe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 -- 172.21.15.145:0/3320095250 >> 172.21.15.145:0/3320095250 conn(0x7f14440fb270 msgr2=0x7f14440fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 -- 172.21.15.145:0/3320095250 shutdown_connections 2023-08-25T04:47:03.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.390+0000 7f1449dff700 1 -- 172.21.15.145:0/3320095250 wait complete. 2023-08-25T04:47:03.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.391+0000 7f1449dff700 1 Processor -- start 2023-08-25T04:47:03.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 -- start start 2023-08-25T04:47:03.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 0x7f14440fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444105770 0x7f14440fc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441a4ea0 con 0x7f1444105770 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441a5010 con 0x7f1444100120 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1449dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441a51d0 con 0x7f14440ff760 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1442ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 0x7f14440fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53566/0 (socket says 172.21.15.145:53566) 2023-08-25T04:47:03.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f1442ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 0x7f14440fbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54280/0 (socket says 172.21.15.145:54280) 2023-08-25T04:47:03.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.392+0000 7f14437fe700 1 -- 172.21.15.145:0/2317294868 learned_addr learned my addr 172.21.15.145:0/2317294868 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:03.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f1443fff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444105770 0x7f14440fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 -- 172.21.15.145:0/2317294868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 msgr2=0x7f14440fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 0x7f14440fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 -- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444105770 msgr2=0x7f14440fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444105770 0x7f14440fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 -- 172.21.15.145:0/2317294868 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438009520 con 0x7f14440ff760 2023-08-25T04:47:03.397 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f14437fe700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f143400e9c0 tx=0x7f143400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f1440ff9700 1 -- 172.21.15.145:0/2317294868 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400cbd0 con 0x7f14440ff760 2023-08-25T04:47:03.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f1440ff9700 1 -- 172.21.15.145:0/2317294868 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f143400eeb0 con 0x7f14440ff760 2023-08-25T04:47:03.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.393+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441a54b0 con 0x7f14440ff760 2023-08-25T04:47:03.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.394+0000 7f1440ff9700 1 -- 172.21.15.145:0/2317294868 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434005580 con 0x7f14440ff760 2023-08-25T04:47:03.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.394+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14441a59e0 con 0x7f14440ff760 2023-08-25T04:47:03.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.398+0000 7f1440ff9700 1 -- 172.21.15.145:0/2317294868 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f1434004750 con 0x7f14440ff760 2023-08-25T04:47:03.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.400+0000 7f145d70eb80 1 Processor -- start 2023-08-25T04:47:03.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.400+0000 7f145d70eb80 1 -- start start 2023-08-25T04:47:03.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.400+0000 7f1440ff9700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 0x7f142c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f1440ff9700 1 -- 172.21.15.145:0/2317294868 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1434014070 con 0x7f14440ff760 2023-08-25T04:47:03.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351320c7b0 con 0x5635132aaca0 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635131fd780 con 0x5635132aa380 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f145d70eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635131fb0a0 con 0x5635132a4c40 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f1443fff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 0x7f142c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54296/0 (socket says 172.21.15.145:54296) 2023-08-25T04:47:03.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 -- 172.21.15.145:0/1976672326 learned_addr learned my addr 172.21.15.145:0/1976672326 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:03.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f1443fff700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f1442ffd700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 -- 172.21.15.145:0/1976672326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 msgr2=0x5635132a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.404 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 -- 172.21.15.145:0/1976672326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 msgr2=0x5635132a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f14437fe700 1 -- 172.21.15.145:0/1976672326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563513076630 con 0x5635132aa380 2023-08-25T04:47:03.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.401+0000 7f1443fff700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:47:03.405 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.402+0000 7f14437fe700 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f143400c5b0 tx=0x7f143400cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb9dd64600633b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.402+0000 7f1443fff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 0x7f142c0a2600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f143c009510 tx=0x7f143c009920 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.402+0000 7f1429ffb700 1 -- 172.21.15.145:0/1976672326 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434098e20 con 0x5635132aa380 2023-08-25T04:47:03.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.402+0000 7f1429ffb700 1 -- 172.21.15.145:0/1976672326 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1434098720 con 0x5635132aa380 2023-08-25T04:47:03.406 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.402+0000 7f1429ffb700 1 -- 172.21.15.145:0/1976672326 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434096be0 con 0x5635132aa380 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 -- 172.21.15.145:0/1976672326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 msgr2=0x5635132aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f143400c5b0 tx=0x7f143400cf10 comp rx=0 tx=0).stop 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 -- 172.21.15.145:0/1976672326 shutdown_connections 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132aa760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 --2- 172.21.15.145:0/1976672326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 -- 172.21.15.145:0/1976672326 >> 172.21.15.145:0/1976672326 conn(0x5635132190e0 msgr2=0x56351329f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 -- 172.21.15.145:0/1976672326 shutdown_connections 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.403+0000 7f145d70eb80 1 -- 172.21.15.145:0/1976672326 wait complete. 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.404+0000 7f145d70eb80 1 -- start start 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.404+0000 7f145d70eb80 1 Processor -- start 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132aaca0 0x5635132ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635131f6990 con 0x5635132a4c40 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563513194bb0 con 0x5635132aa380 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635131948b0 con 0x5635132aaca0 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f145d70eb80 10 client.0 _ll_register_callbacks cb 0x5635130dea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f1442ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f14437fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f14437fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37886/0 (socket says 172.21.15.145:37886) 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f14437fe700 1 -- 172.21.15.145:0/3064794018 learned_addr learned my addr 172.21.15.145:0/3064794018 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.405+0000 7f1443fff700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132aaca0 0x5635132ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 -- 172.21.15.145:0/3064794018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132aaca0 msgr2=0x5635132ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132aaca0 0x5635132ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 -- 172.21.15.145:0/3064794018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 msgr2=0x5635132adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 -- 172.21.15.145:0/3064794018 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14441a59e0 con 0x5635132a4c40 2023-08-25T04:47:03.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f1442ffd700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:47:03.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f14437fe700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14340c8ce0 tx=0x7f1434005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3ff70f03ee30f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f1422ffd700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:47:03.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143409c380 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f143409c550 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.406+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635132741c0 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14340b62c0 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 10 client.15798 fetch_fsmap learned FSMap version 27 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563513022de0 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635132741c0 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f14340a00a0 con 0x5635132a4c40 2023-08-25T04:47:03.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 10 client.15798 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 20 client.15798 populate_metadata read hostname 'smithi145' 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635132b38a0 0x5635132b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 4 client.15798 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.407+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56351319cba0 con 0x5635132b38a0 2023-08-25T04:47:03.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.408+0000 7f1442ffd700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635132b38a0 0x5635132b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.408+0000 7f1442ffd700 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635132b38a0 0x5635132b5cc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73b57a6b50548f3 server_cookie=d6312cf19439fe81 in_seq=0 out_seq=0 2023-08-25T04:47:03.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.408+0000 7f1422ffd700 10 client.15798 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:03.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:02 smithi162 ceph-mon[130941]: pgmap v62: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:03.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f1422ffd700 1 -- 172.21.15.145:0/3064794018 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56351319cba0 con 0x5635132b38a0 2023-08-25T04:47:03.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f1422ffd700 10 client.15798 handle_command_reply: tid=0 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 client.15798 shutdown 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 msgr2=0x5635132b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14340c8ce0 tx=0x7f1434005910 comp rx=0 tx=0).stop 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 shutdown_connections 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635132b38a0 0x5635132b5cc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132aaca0 0x5635132ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132a4c40 0x5635132b0ca0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3064794018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 >> 172.21.15.145:0/3064794018 conn(0x5635132190e0 msgr2=0x563513299b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f1422ffd700 0 client.15798 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 shutdown_connections 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.410+0000 7f145d70eb80 1 -- 172.21.15.145:0/3064794018 wait complete. 2023-08-25T04:47:03.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.411+0000 7f145d70eb80 20 client.15798 trim_cache size 0 max 16384 2023-08-25T04:47:03.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:02 smithi183 ceph-mon[131414]: pgmap v62: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:03.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.438+0000 7f145d70eb80 1 Processor -- start 2023-08-25T04:47:03.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.438+0000 7f145d70eb80 1 -- start start 2023-08-25T04:47:03.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132a9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x563513078920 con 0x5635132aaca0 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351329f9e0 con 0x5635132aa380 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f145d70eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x563513297770 con 0x5635132a4c40 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f1442ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53590/0 (socket says 172.21.15.145:53590) 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.439+0000 7f14437fe700 1 -- 172.21.15.145:0/3981060923 learned_addr learned my addr 172.21.15.145:0/3981060923 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 -- 172.21.15.145:0/3981060923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 msgr2=0x5635132a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 -- 172.21.15.145:0/3981060923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 msgr2=0x5635132b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 -- 172.21.15.145:0/3981060923 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56351300bc60 con 0x5635132a4c40 2023-08-25T04:47:03.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f14437fe700 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14340a06f0 tx=0x7f1434009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41222f8e5238b274 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f142a7fc700 1 -- 172.21.15.145:0/3981060923 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434098e20 con 0x5635132a4c40 2023-08-25T04:47:03.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f142a7fc700 1 -- 172.21.15.145:0/3981060923 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f143409c380 con 0x5635132a4c40 2023-08-25T04:47:03.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.440+0000 7f142a7fc700 1 -- 172.21.15.145:0/3981060923 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340a00a0 con 0x5635132a4c40 2023-08-25T04:47:03.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 -- 172.21.15.145:0/3981060923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 msgr2=0x5635132a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14340a06f0 tx=0x7f1434009550 comp rx=0 tx=0).stop 2023-08-25T04:47:03.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 -- 172.21.15.145:0/3981060923 shutdown_connections 2023-08-25T04:47:03.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aaca0 0x5635132b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132a4700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 --2- 172.21.15.145:0/3981060923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aa380 0x5635132a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 -- 172.21.15.145:0/3981060923 >> 172.21.15.145:0/3981060923 conn(0x5635132190e0 msgr2=0x5635132c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 -- 172.21.15.145:0/3981060923 shutdown_connections 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.441+0000 7f145d70eb80 1 -- 172.21.15.145:0/3981060923 wait complete. 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.442+0000 7f145d70eb80 1 -- start start 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.442+0000 7f145d70eb80 1 Processor -- start 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.442+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.442+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aa380 0x5635132a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.442+0000 7f145d70eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 0x5635132a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f145d70eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635132a7920 con 0x5635132aa380 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f145d70eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635132a7a90 con 0x5635132aaca0 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f145d70eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635132ac800 con 0x5635132a4c40 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f145d70eb80 10 client.0 _ll_register_callbacks cb 0x5635130dea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53592/0 (socket says 172.21.15.145:53592) 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 -- 172.21.15.145:0/2961487499 learned_addr learned my addr 172.21.15.145:0/2961487499 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:03.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f1442ffd700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aa380 0x5635132a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f1443fff700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 0x5635132a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 -- 172.21.15.145:0/2961487499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 msgr2=0x5635132a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 0x5635132a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 -- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aa380 msgr2=0x5635132a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aa380 0x5635132a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 -- 172.21.15.145:0/2961487499 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563513022de0 con 0x5635132a4c40 2023-08-25T04:47:03.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f1443fff700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 0x5635132a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f14437fe700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1434098420 tx=0x7f1434096cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad8bfd4ccd1e02c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.443+0000 7f1422ffd700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434098e20 con 0x5635132a4c40 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635132978b0 con 0x5635132a4c40 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f14340a0530 con 0x5635132a4c40 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1434077340 con 0x5635132a4c40 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 10 client.25822 fetch_fsmap learned FSMap version 27 2023-08-25T04:47:03.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563513049250 con 0x5635132a4c40 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635132978b0 con 0x5635132a4c40 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1434094510 con 0x5635132a4c40 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 10 client.25822 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 10 client.25822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 20 client.25822 populate_metadata read hostname 'smithi145' 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635133128e0 0x563513314cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:03.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 4 client.25822 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.444+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635131ef970 con 0x5635133128e0 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.445+0000 7f1442ffd700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635133128e0 0x563513314cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.445+0000 7f1442ffd700 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635133128e0 0x563513314cd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f7b4f3c5343847 server_cookie=8ff27833af9b7895 in_seq=0 out_seq=0 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.445+0000 7f1422ffd700 10 client.25822 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f1422ffd700 1 -- 172.21.15.145:0/2961487499 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5635131ef970 con 0x5635133128e0 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f1422ffd700 10 client.25822 handle_command_reply: tid=0 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 client.25822 shutdown 2023-08-25T04:47:03.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 msgr2=0x5635132c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1434098420 tx=0x7f1434096cd0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 shutdown_connections 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635132aaca0 0x5635132a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5635133128e0 0x563513314cd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635132a4c40 0x5635132c85d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 --2- 172.21.15.145:0/2961487499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5635132aa380 0x5635132a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 >> 172.21.15.145:0/2961487499 conn(0x5635132190e0 msgr2=0x5635132992b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f1422ffd700 0 client.25822 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:03.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 shutdown_connections 2023-08-25T04:47:03.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.446+0000 7f145d70eb80 1 -- 172.21.15.145:0/2961487499 wait complete. 2023-08-25T04:47:03.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.447+0000 7f145d70eb80 20 client.25822 trim_cache size 0 max 16384 2023-08-25T04:47:03.456 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:47:03.457 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:47:03.458 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:47:03.458 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:47:03.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 msgr2=0x7f142c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 0x7f142c0a2600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f143c009510 tx=0x7f143c009920 comp rx=0 tx=0).stop 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 msgr2=0x7f14440fb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f143400e9c0 tx=0x7f143400ecd0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 shutdown_connections 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1444105770 0x7f14440fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14440ff760 0x7f14440fb9d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f142c0a0140 0x7f142c0a2600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 --2- 172.21.15.145:0/2317294868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1444100120 0x7f14440fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:03.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 >> 172.21.15.145:0/2317294868 conn(0x7f14440fb270 msgr2=0x7f1444067c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:03.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 shutdown_connections 2023-08-25T04:47:03.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:03.448+0000 7f1449dff700 1 -- 172.21.15.145:0/2317294868 wait complete. 2023-08-25T04:47:03.470 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:47:05.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:05 smithi162 ceph-mon[130941]: pgmap v63: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:05.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:05 smithi183 ceph-mon[131414]: pgmap v63: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:05 smithi145 ceph-mon[97681]: pgmap v63: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 79 MiB/s wr, 102 op/s 2023-08-25T04:47:07.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:07 smithi162 ceph-mon[130941]: pgmap v64: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 80 MiB/s wr, 238 op/s 2023-08-25T04:47:07.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:07 smithi183 ceph-mon[131414]: pgmap v64: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 80 MiB/s wr, 238 op/s 2023-08-25T04:47:08.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:07 smithi145 ceph-mon[97681]: pgmap v64: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 80 MiB/s wr, 238 op/s 2023-08-25T04:47:09.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:09 smithi162 ceph-mon[130941]: pgmap v65: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 216 op/s 2023-08-25T04:47:09.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:09 smithi183 ceph-mon[131414]: pgmap v65: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 216 op/s 2023-08-25T04:47:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:09 smithi145 ceph-mon[97681]: pgmap v65: 129 pgs: 129 active+clean; 6.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 216 op/s 2023-08-25T04:47:10.955 INFO:teuthology.orchestra.run.smithi145.stderr:35300000 of 50000000 tuples (70%) done (elapsed 108.42 s, remaining 45.15 s) 2023-08-25T04:47:11.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:11 smithi162 ceph-mon[130941]: pgmap v66: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 355 op/s 2023-08-25T04:47:11.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:11 smithi183 ceph-mon[131414]: pgmap v66: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 355 op/s 2023-08-25T04:47:12.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:11 smithi145 ceph-mon[97681]: pgmap v66: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 355 op/s 2023-08-25T04:47:13.199 INFO:teuthology.orchestra.run.smithi145.stderr:35400000 of 50000000 tuples (70%) done (elapsed 110.67 s, remaining 45.64 s) 2023-08-25T04:47:13.399 INFO:teuthology.orchestra.run.smithi145.stderr:35500000 of 50000000 tuples (71%) done (elapsed 110.87 s, remaining 45.28 s) 2023-08-25T04:47:13.631 INFO:teuthology.orchestra.run.smithi145.stderr:35600000 of 50000000 tuples (71%) done (elapsed 111.10 s, remaining 44.94 s) 2023-08-25T04:47:13.797 INFO:teuthology.orchestra.run.smithi145.stderr:35700000 of 50000000 tuples (71%) done (elapsed 111.27 s, remaining 44.57 s) 2023-08-25T04:47:13.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:13 smithi162 ceph-mon[130941]: pgmap v67: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:13.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:13 smithi183 ceph-mon[131414]: pgmap v67: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:13.949 INFO:teuthology.orchestra.run.smithi145.stderr:35800000 of 50000000 tuples (71%) done (elapsed 111.42 s, remaining 44.19 s) 2023-08-25T04:47:14.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:13 smithi145 ceph-mon[97681]: pgmap v67: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:14.181 INFO:teuthology.orchestra.run.smithi145.stderr:35900000 of 50000000 tuples (71%) done (elapsed 111.65 s, remaining 43.85 s) 2023-08-25T04:47:14.403 INFO:teuthology.orchestra.run.smithi145.stderr:36000000 of 50000000 tuples (72%) done (elapsed 111.87 s, remaining 43.51 s) 2023-08-25T04:47:14.615 INFO:teuthology.orchestra.run.smithi145.stderr:36100000 of 50000000 tuples (72%) done (elapsed 112.08 s, remaining 43.16 s) 2023-08-25T04:47:14.813 INFO:teuthology.orchestra.run.smithi145.stderr:36200000 of 50000000 tuples (72%) done (elapsed 112.28 s, remaining 42.80 s) 2023-08-25T04:47:14.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:47:14.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.912 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:14 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:47:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:14 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:14.951 INFO:teuthology.orchestra.run.smithi145.stderr:36300000 of 50000000 tuples (72%) done (elapsed 112.42 s, remaining 42.43 s) 2023-08-25T04:47:15.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:47:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:15.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:14 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:15.189 INFO:teuthology.orchestra.run.smithi145.stderr:36400000 of 50000000 tuples (72%) done (elapsed 112.66 s, remaining 42.09 s) 2023-08-25T04:47:15.395 INFO:teuthology.orchestra.run.smithi145.stderr:36500000 of 50000000 tuples (73%) done (elapsed 112.87 s, remaining 41.74 s) 2023-08-25T04:47:15.608 INFO:teuthology.orchestra.run.smithi145.stderr:36600000 of 50000000 tuples (73%) done (elapsed 113.08 s, remaining 41.40 s) 2023-08-25T04:47:15.849 INFO:teuthology.orchestra.run.smithi145.stderr:36700000 of 50000000 tuples (73%) done (elapsed 113.32 s, remaining 41.07 s) 2023-08-25T04:47:15.992 INFO:teuthology.orchestra.run.smithi145.stderr:36800000 of 50000000 tuples (73%) done (elapsed 113.46 s, remaining 40.70 s) 2023-08-25T04:47:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:15 smithi162 ceph-mon[130941]: pgmap v68: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:15 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:15 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:15 smithi183 ceph-mon[131414]: pgmap v68: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:15 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:15 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.217 INFO:teuthology.orchestra.run.smithi145.stderr:36900000 of 50000000 tuples (73%) done (elapsed 113.69 s, remaining 40.36 s) 2023-08-25T04:47:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:15 smithi145 ceph-mon[97681]: pgmap v68: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2023-08-25T04:47:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:15 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:15 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:16.395 INFO:teuthology.orchestra.run.smithi145.stderr:37000000 of 50000000 tuples (74%) done (elapsed 113.86 s, remaining 40.01 s) 2023-08-25T04:47:16.735 INFO:teuthology.orchestra.run.smithi145.stderr:37100000 of 50000000 tuples (74%) done (elapsed 114.20 s, remaining 39.71 s) 2023-08-25T04:47:17.124 INFO:teuthology.orchestra.run.smithi145.stderr:37200000 of 50000000 tuples (74%) done (elapsed 114.59 s, remaining 39.43 s) 2023-08-25T04:47:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:16 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:47:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:16 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:47:17.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:16 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:16 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:47:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:16 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:47:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:16 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:17.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:16 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:47:17.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:16 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:47:17.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:16 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:47:17.575 INFO:teuthology.orchestra.run.smithi145.stderr:37300000 of 50000000 tuples (74%) done (elapsed 115.05 s, remaining 39.17 s) 2023-08-25T04:47:18.303 INFO:teuthology.orchestra.run.smithi145.stderr:37400000 of 50000000 tuples (74%) done (elapsed 115.77 s, remaining 39.00 s) 2023-08-25T04:47:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:17 smithi145 ceph-mon[97681]: pgmap v69: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 352 op/s 2023-08-25T04:47:18.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:17 smithi162 ceph-mon[130941]: pgmap v69: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 352 op/s 2023-08-25T04:47:18.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:17 smithi183 ceph-mon[131414]: pgmap v69: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 352 op/s 2023-08-25T04:47:18.516 INFO:teuthology.orchestra.run.smithi145.stderr:37500000 of 50000000 tuples (75%) done (elapsed 115.99 s, remaining 38.66 s) 2023-08-25T04:47:18.702 INFO:teuthology.orchestra.run.smithi145.stderr:37600000 of 50000000 tuples (75%) done (elapsed 116.17 s, remaining 38.31 s) 2023-08-25T04:47:18.842 INFO:teuthology.orchestra.run.smithi145.stderr:37700000 of 50000000 tuples (75%) done (elapsed 116.31 s, remaining 37.95 s) 2023-08-25T04:47:19.049 INFO:teuthology.orchestra.run.smithi145.stderr:37800000 of 50000000 tuples (75%) done (elapsed 116.52 s, remaining 37.61 s) 2023-08-25T04:47:19.252 INFO:teuthology.orchestra.run.smithi145.stderr:37900000 of 50000000 tuples (75%) done (elapsed 116.72 s, remaining 37.26 s) 2023-08-25T04:47:19.489 INFO:teuthology.orchestra.run.smithi145.stderr:38000000 of 50000000 tuples (76%) done (elapsed 116.96 s, remaining 36.93 s) 2023-08-25T04:47:19.684 INFO:teuthology.orchestra.run.smithi145.stderr:38100000 of 50000000 tuples (76%) done (elapsed 117.15 s, remaining 36.59 s) 2023-08-25T04:47:19.881 INFO:teuthology.orchestra.run.smithi145.stderr:38200000 of 50000000 tuples (76%) done (elapsed 117.35 s, remaining 36.25 s) 2023-08-25T04:47:20.019 INFO:teuthology.orchestra.run.smithi145.stderr:38300000 of 50000000 tuples (76%) done (elapsed 117.49 s, remaining 35.89 s) 2023-08-25T04:47:20.211 INFO:teuthology.orchestra.run.smithi145.stderr:38400000 of 50000000 tuples (76%) done (elapsed 117.68 s, remaining 35.55 s) 2023-08-25T04:47:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:19 smithi145 ceph-mon[97681]: pgmap v70: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 216 op/s 2023-08-25T04:47:20.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:19 smithi162 ceph-mon[130941]: pgmap v70: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 216 op/s 2023-08-25T04:47:20.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:19 smithi183 ceph-mon[131414]: pgmap v70: 129 pgs: 129 active+clean; 6.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 216 op/s 2023-08-25T04:47:20.443 INFO:teuthology.orchestra.run.smithi145.stderr:38500000 of 50000000 tuples (77%) done (elapsed 117.91 s, remaining 35.22 s) 2023-08-25T04:47:20.659 INFO:teuthology.orchestra.run.smithi145.stderr:38600000 of 50000000 tuples (77%) done (elapsed 118.13 s, remaining 34.89 s) 2023-08-25T04:47:20.856 INFO:teuthology.orchestra.run.smithi145.stderr:38700000 of 50000000 tuples (77%) done (elapsed 118.33 s, remaining 34.55 s) 2023-08-25T04:47:20.997 INFO:teuthology.orchestra.run.smithi145.stderr:38800000 of 50000000 tuples (77%) done (elapsed 118.47 s, remaining 34.20 s) 2023-08-25T04:47:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:38900000 of 50000000 tuples (77%) done (elapsed 118.81 s, remaining 33.90 s) 2023-08-25T04:47:21.997 INFO:teuthology.orchestra.run.smithi145.stderr:39000000 of 50000000 tuples (78%) done (elapsed 119.47 s, remaining 33.70 s) 2023-08-25T04:47:22.102 INFO:teuthology.orchestra.run.smithi145.stderr:39100000 of 50000000 tuples (78%) done (elapsed 119.57 s, remaining 33.33 s) 2023-08-25T04:47:22.293 INFO:teuthology.orchestra.run.smithi145.stderr:39200000 of 50000000 tuples (78%) done (elapsed 119.76 s, remaining 33.00 s) 2023-08-25T04:47:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:21 smithi145 ceph-mon[97681]: pgmap v71: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 71 MiB/s wr, 248 op/s 2023-08-25T04:47:22.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:21 smithi162 ceph-mon[130941]: pgmap v71: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 71 MiB/s wr, 248 op/s 2023-08-25T04:47:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:21 smithi183 ceph-mon[131414]: pgmap v71: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 71 MiB/s wr, 248 op/s 2023-08-25T04:47:22.515 INFO:teuthology.orchestra.run.smithi145.stderr:39300000 of 50000000 tuples (78%) done (elapsed 119.98 s, remaining 32.67 s) 2023-08-25T04:47:22.669 INFO:teuthology.orchestra.run.smithi145.stderr:39400000 of 50000000 tuples (78%) done (elapsed 120.14 s, remaining 32.32 s) 2023-08-25T04:47:22.876 INFO:teuthology.orchestra.run.smithi145.stderr:39500000 of 50000000 tuples (79%) done (elapsed 120.35 s, remaining 31.99 s) 2023-08-25T04:47:23.112 INFO:teuthology.orchestra.run.smithi145.stderr:39600000 of 50000000 tuples (79%) done (elapsed 120.58 s, remaining 31.67 s) 2023-08-25T04:47:23.287 INFO:teuthology.orchestra.run.smithi145.stderr:39700000 of 50000000 tuples (79%) done (elapsed 120.76 s, remaining 31.33 s) 2023-08-25T04:47:23.490 INFO:teuthology.orchestra.run.smithi145.stderr:39800000 of 50000000 tuples (79%) done (elapsed 120.96 s, remaining 31.00 s) 2023-08-25T04:47:23.633 INFO:teuthology.orchestra.run.smithi145.stderr:39900000 of 50000000 tuples (79%) done (elapsed 121.10 s, remaining 30.66 s) 2023-08-25T04:47:23.862 INFO:teuthology.orchestra.run.smithi145.stderr:40000000 of 50000000 tuples (80%) done (elapsed 121.33 s, remaining 30.33 s) 2023-08-25T04:47:24.045 INFO:teuthology.orchestra.run.smithi145.stderr:40100000 of 50000000 tuples (80%) done (elapsed 121.52 s, remaining 30.00 s) 2023-08-25T04:47:24.247 INFO:teuthology.orchestra.run.smithi145.stderr:40200000 of 50000000 tuples (80%) done (elapsed 121.72 s, remaining 29.67 s) 2023-08-25T04:47:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:23 smithi145 ceph-mon[97681]: pgmap v72: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:24.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:23 smithi162 ceph-mon[130941]: pgmap v72: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:24.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:23 smithi183 ceph-mon[131414]: pgmap v72: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:24.443 INFO:teuthology.orchestra.run.smithi145.stderr:40300000 of 50000000 tuples (80%) done (elapsed 121.91 s, remaining 29.34 s) 2023-08-25T04:47:24.650 INFO:teuthology.orchestra.run.smithi145.stderr:40400000 of 50000000 tuples (80%) done (elapsed 122.12 s, remaining 29.02 s) 2023-08-25T04:47:24.788 INFO:teuthology.orchestra.run.smithi145.stderr:40500000 of 50000000 tuples (81%) done (elapsed 122.26 s, remaining 28.68 s) 2023-08-25T04:47:25.148 INFO:teuthology.orchestra.run.smithi145.stderr:40600000 of 50000000 tuples (81%) done (elapsed 122.62 s, remaining 28.39 s) 2023-08-25T04:47:25.616 INFO:teuthology.orchestra.run.smithi145.stderr:40700000 of 50000000 tuples (81%) done (elapsed 123.09 s, remaining 28.13 s) 2023-08-25T04:47:25.938 INFO:teuthology.orchestra.run.smithi145.stderr:40800000 of 50000000 tuples (81%) done (elapsed 123.41 s, remaining 27.83 s) 2023-08-25T04:47:26.114 INFO:teuthology.orchestra.run.smithi145.stderr:40900000 of 50000000 tuples (81%) done (elapsed 123.58 s, remaining 27.50 s) 2023-08-25T04:47:26.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:25 smithi145 ceph-mon[97681]: pgmap v73: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:26.350 INFO:teuthology.orchestra.run.smithi145.stderr:41000000 of 50000000 tuples (82%) done (elapsed 123.82 s, remaining 27.18 s) 2023-08-25T04:47:26.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:25 smithi162 ceph-mon[130941]: pgmap v73: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:26.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:25 smithi183 ceph-mon[131414]: pgmap v73: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 70 MiB/s wr, 109 op/s 2023-08-25T04:47:26.782 INFO:teuthology.orchestra.run.smithi145.stderr:41100000 of 50000000 tuples (82%) done (elapsed 124.25 s, remaining 26.91 s) 2023-08-25T04:47:27.174 INFO:teuthology.orchestra.run.smithi145.stderr:41200000 of 50000000 tuples (82%) done (elapsed 124.64 s, remaining 26.62 s) 2023-08-25T04:47:27.336 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:27 smithi145 ceph-mon[97681]: pgmap v74: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 89 MiB/s wr, 155 op/s 2023-08-25T04:47:27.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:27 smithi162 ceph-mon[130941]: pgmap v74: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 89 MiB/s wr, 155 op/s 2023-08-25T04:47:27.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:27 smithi183 ceph-mon[131414]: pgmap v74: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 89 MiB/s wr, 155 op/s 2023-08-25T04:47:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:41300000 of 50000000 tuples (82%) done (elapsed 125.20 s, remaining 26.37 s) 2023-08-25T04:47:28.275 INFO:teuthology.orchestra.run.smithi145.stderr:41400000 of 50000000 tuples (82%) done (elapsed 125.74 s, remaining 26.12 s) 2023-08-25T04:47:28.417 INFO:teuthology.orchestra.run.smithi145.stderr:41500000 of 50000000 tuples (83%) done (elapsed 125.89 s, remaining 25.78 s) 2023-08-25T04:47:28.647 INFO:teuthology.orchestra.run.smithi145.stderr:41600000 of 50000000 tuples (83%) done (elapsed 126.12 s, remaining 25.47 s) 2023-08-25T04:47:28.884 INFO:teuthology.orchestra.run.smithi145.stderr:41700000 of 50000000 tuples (83%) done (elapsed 126.35 s, remaining 25.15 s) 2023-08-25T04:47:29.060 INFO:teuthology.orchestra.run.smithi145.stderr:41800000 of 50000000 tuples (83%) done (elapsed 126.53 s, remaining 24.82 s) 2023-08-25T04:47:29.408 INFO:teuthology.orchestra.run.smithi145.stderr:PQputline failed 2023-08-25T04:47:29.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T04:47:29.417 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi145 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i'" 2023-08-25T04:47:29.525 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5b701f5cda6a482b84dec29cd095950d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi145 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i'" 2023-08-25T04:47:29.527 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-08-25T04:47:29.538 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-08-25T04:47:29.539 INFO:teuthology.task.exec:Executing custom commands... 2023-08-25T04:47:29.539 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T04:47:29.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-08-25T04:47:29.692 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:29 smithi145 ceph-mon[97681]: pgmap v75: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 79 MiB/s wr, 78 op/s 2023-08-25T04:47:29.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-08-25T04:47:29.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:29 smithi162 ceph-mon[130941]: pgmap v75: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 79 MiB/s wr, 78 op/s 2023-08-25T04:47:29.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:29 smithi183 ceph-mon[131414]: pgmap v75: 129 pgs: 129 active+clean; 7.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 79 MiB/s wr, 78 op/s 2023-08-25T04:47:31.911 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:31 smithi162 ceph-mon[130941]: pgmap v76: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 142 MiB/s wr, 208 op/s 2023-08-25T04:47:31.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:31 smithi183 ceph-mon[131414]: pgmap v76: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 142 MiB/s wr, 208 op/s 2023-08-25T04:47:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:31 smithi145 ceph-mon[97681]: pgmap v76: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 142 MiB/s wr, 208 op/s 2023-08-25T04:47:33.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:47:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 -- 172.21.15.145:0/2838370786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c100870 msgr2=0x7fc96c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 --2- 172.21.15.145:0/2838370786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c100870 0x7fc96c100cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 -- 172.21.15.145:0/2838370786 shutdown_connections 2023-08-25T04:47:33.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 --2- 172.21.15.145:0/2838370786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c101230 0x7fc96c10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 --2- 172.21.15.145:0/2838370786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c106880 0x7fc96c106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 --2- 172.21.15.145:0/2838370786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c100870 0x7fc96c100cf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 -- 172.21.15.145:0/2838370786 >> 172.21.15.145:0/2838370786 conn(0x7fc96c0fc3a0 msgr2=0x7fc96c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 -- 172.21.15.145:0/2838370786 shutdown_connections 2023-08-25T04:47:33.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.716+0000 7fc9741ff700 1 -- 172.21.15.145:0/2838370786 wait complete. 2023-08-25T04:47:33.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.717+0000 7fc9741ff700 1 Processor -- start 2023-08-25T04:47:33.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.717+0000 7fc9741ff700 1 -- start start 2023-08-25T04:47:33.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c100870 0x7fc96c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c101230 0x7fc96c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c199060 con 0x7fc96c101230 2023-08-25T04:47:33.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1991d0 con 0x7fc96c100870 2023-08-25T04:47:33.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc9741ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c199390 con 0x7fc96c106880 2023-08-25T04:47:33.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:46076/0 (socket says 172.21.15.145:46076) 2023-08-25T04:47:33.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 -- 172.21.15.145:0/1998061465 learned_addr learned my addr 172.21.15.145:0/1998061465 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:33.732 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c100870 0x7fc96c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97179a700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c101230 0x7fc96c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 -- 172.21.15.145:0/1998061465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c100870 msgr2=0x7fc96c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c100870 0x7fc96c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 -- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c101230 msgr2=0x7fc96c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c101230 0x7fc96c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.718+0000 7fc97279c700 1 -- 172.21.15.145:0/1998061465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c0095f0 con 0x7fc96c106880 2023-08-25T04:47:33.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc97279c700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc96400ef30 tx=0x7fc96400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc962ffd700 1 -- 172.21.15.145:0/1998061465 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964018070 con 0x7fc96c106880 2023-08-25T04:47:33.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc962ffd700 1 -- 172.21.15.145:0/1998061465 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc96400f040 con 0x7fc96c106880 2023-08-25T04:47:33.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c199610 con 0x7fc96c106880 2023-08-25T04:47:33.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc962ffd700 1 -- 172.21.15.145:0/1998061465 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400cd00 con 0x7fc96c106880 2023-08-25T04:47:33.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.719+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c1a5970 con 0x7fc96c106880 2023-08-25T04:47:33.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.723+0000 7fc962ffd700 1 -- 172.21.15.145:0/1998061465 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fc964003a00 con 0x7fc96c106880 2023-08-25T04:47:33.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.724+0000 7fc962ffd700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 0x7fc9580a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.724+0000 7fc962ffd700 1 -- 172.21.15.145:0/1998061465 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc9640cb400 con 0x7fc96c106880 2023-08-25T04:47:33.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.725+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 0x7fc9580a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.725+0000 7fc987b5bb80 1 Processor -- start 2023-08-25T04:47:33.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.725+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 0x7fc9580a2600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc968009be0 tx=0x7fc968009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.725+0000 7fc987b5bb80 1 -- start start 2023-08-25T04:47:33.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce893bc40 0x559ce893c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce88a37b0 con 0x559ce8941380 2023-08-25T04:47:33.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8894780 con 0x559ce8941ca0 2023-08-25T04:47:33.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce88920a0 con 0x559ce893bc40 2023-08-25T04:47:33.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc971f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc971f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46496/0 (socket says 172.21.15.145:46496) 2023-08-25T04:47:33.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc971f9b700 1 -- 172.21.15.145:0/1247710951 learned_addr learned my addr 172.21.15.145:0/1247710951 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc97179a700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc97279c700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce893bc40 0x559ce893c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc971f9b700 1 -- 172.21.15.145:0/1247710951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce893bc40 msgr2=0x559ce893c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.726+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce893bc40 0x559ce893c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc971f9b700 1 -- 172.21.15.145:0/1247710951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 msgr2=0x559ce893b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc971f9b700 1 -- 172.21.15.145:0/1247710951 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce870d630 con 0x559ce8941380 2023-08-25T04:47:33.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc97179a700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:47:33.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc968011d70 tx=0x7fc968014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ac678668d01636 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc957fff700 1 -- 172.21.15.145:0/1247710951 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011da0 con 0x559ce8941380 2023-08-25T04:47:33.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc957fff700 1 -- 172.21.15.145:0/1247710951 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc968004020 con 0x559ce8941380 2023-08-25T04:47:33.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.727+0000 7fc957fff700 1 -- 172.21.15.145:0/1247710951 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968003cb0 con 0x559ce8941380 2023-08-25T04:47:33.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1247710951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 msgr2=0x559ce8941760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc968011d70 tx=0x7fc968014000 comp rx=0 tx=0).stop 2023-08-25T04:47:33.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1247710951 shutdown_connections 2023-08-25T04:47:33.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce893bc40 0x559ce893c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce8941380 0x559ce8941760 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1247710951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1247710951 >> 172.21.15.145:0/1247710951 conn(0x559ce88b00e0 msgr2=0x559ce8936410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1247710951 shutdown_connections 2023-08-25T04:47:33.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.728+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1247710951 wait complete. 2023-08-25T04:47:33.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 -- start start 2023-08-25T04:47:33.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 Processor -- start 2023-08-25T04:47:33.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8947c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce8945670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce888d990 con 0x559ce893bc40 2023-08-25T04:47:33.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce882bbb0 con 0x559ce8941ca0 2023-08-25T04:47:33.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.729+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce882b8b0 con 0x559ce8941380 2023-08-25T04:47:33.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc987b5bb80 10 client.0 _ll_register_callbacks cb 0x559ce8775a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:47:33.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:46080/0 (socket says 172.21.15.145:46080) 2023-08-25T04:47:33.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 -- 172.21.15.145:0/673788313 learned_addr learned my addr 172.21.15.145:0/673788313 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:33.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc971f9b700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8947c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97279c700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce8945670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 -- 172.21.15.145:0/673788313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 msgr2=0x559ce8945670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce8945670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 -- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 msgr2=0x559ce8947c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8947c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 -- 172.21.15.145:0/673788313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c10e040 con 0x559ce8941380 2023-08-25T04:47:33.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc971f9b700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8947c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:47:33.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc97179a700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d601a612b83a49 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc9557fa700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:47:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.730+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d070 con 0x559ce8941380 2023-08-25T04:47:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce890b1c0 con 0x559ce8941380 2023-08-25T04:47:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc95c004700 con 0x559ce8941380 2023-08-25T04:47:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c0056f0 con 0x559ce8941380 2023-08-25T04:47:33.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 10 client.25834 fetch_fsmap learned FSMap version 27 2023-08-25T04:47:33.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc968017070 con 0x559ce8941380 2023-08-25T04:47:33.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ce890b1c0 con 0x559ce8941380 2023-08-25T04:47:33.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc95c00fa00 con 0x559ce8941380 2023-08-25T04:47:33.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 10 client.25834 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:47:33.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:47:33.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 20 client.25834 populate_metadata read hostname 'smithi145' 2023-08-25T04:47:33.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89556f0 0x559ce8957b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 4 client.25834 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:47:33.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.731+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ce8833ba0 con 0x559ce89556f0 2023-08-25T04:47:33.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.732+0000 7fc971f9b700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89556f0 0x559ce8957b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.733+0000 7fc971f9b700 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89556f0 0x559ce8957b10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7310fe1df4f48cf2 server_cookie=d84815f5e816a973 in_seq=0 out_seq=0 2023-08-25T04:47:33.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.733+0000 7fc9557fa700 10 client.25834 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:33.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc9557fa700 1 -- 172.21.15.145:0/673788313 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ce8833ba0 con 0x559ce89556f0 2023-08-25T04:47:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc9557fa700 10 client.25834 handle_command_reply: tid=0 2023-08-25T04:47:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 client.25834 shutdown 2023-08-25T04:47:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 msgr2=0x559ce8944f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c005f00 comp rx=0 tx=0).stop 2023-08-25T04:47:33.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 shutdown_connections 2023-08-25T04:47:33.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce8945670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8947c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89556f0 0x559ce8957b10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/673788313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8944f30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc9557fa700 0 client.25834 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:33.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 >> 172.21.15.145:0/673788313 conn(0x559ce88b00e0 msgr2=0x559ce8930b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 shutdown_connections 2023-08-25T04:47:33.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 1 -- 172.21.15.145:0/673788313 wait complete. 2023-08-25T04:47:33.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.734+0000 7fc987b5bb80 20 client.25834 trim_cache size 0 max 16384 2023-08-25T04:47:33.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.761+0000 7fc987b5bb80 1 Processor -- start 2023-08-25T04:47:33.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 -- start start 2023-08-25T04:47:33.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce89360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce870f920 con 0x559ce893bc40 2023-08-25T04:47:33.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8937ba0 con 0x559ce8941ca0 2023-08-25T04:47:33.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8932c80 con 0x559ce8941380 2023-08-25T04:47:33.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97179a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97179a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:46096/0 (socket says 172.21.15.145:46096) 2023-08-25T04:47:33.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97179a700 1 -- 172.21.15.145:0/1625942302 learned_addr learned my addr 172.21.15.145:0/1625942302 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:33.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc971f9b700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce89360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97279c700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97179a700 1 -- 172.21.15.145:0/1625942302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 msgr2=0x559ce893f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.762+0000 7fc97179a700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc97179a700 1 -- 172.21.15.145:0/1625942302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 msgr2=0x559ce89360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc97179a700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce89360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc97179a700 1 -- 172.21.15.145:0/1625942302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce895c740 con 0x559ce8941380 2023-08-25T04:47:33.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc97279c700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:47:33.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc97179a700 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0098e0 tx=0x7fc95c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e35b3fb96a2169d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc960ff9700 1 -- 172.21.15.145:0/1625942302 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d070 con 0x559ce8941380 2023-08-25T04:47:33.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc960ff9700 1 -- 172.21.15.145:0/1625942302 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc95c01e040 con 0x559ce8941380 2023-08-25T04:47:33.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.763+0000 7fc960ff9700 1 -- 172.21.15.145:0/1625942302 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0248e0 con 0x559ce8941380 2023-08-25T04:47:33.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1625942302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 msgr2=0x559ce893f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0098e0 tx=0x7fc95c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1625942302 shutdown_connections 2023-08-25T04:47:33.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce893f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce89360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/1625942302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce893f0f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1625942302 >> 172.21.15.145:0/1625942302 conn(0x559ce88b00e0 msgr2=0x559ce8951a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1625942302 shutdown_connections 2023-08-25T04:47:33.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- 172.21.15.145:0/1625942302 wait complete. 2023-08-25T04:47:33.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.764+0000 7fc987b5bb80 1 -- start start 2023-08-25T04:47:33.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 Processor -- start 2023-08-25T04:47:33.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8948cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce89a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8949460 con 0x559ce893bc40 2023-08-25T04:47:33.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce89495d0 con 0x559ce8941ca0 2023-08-25T04:47:33.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce89a8a80 con 0x559ce8941380 2023-08-25T04:47:33.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc987b5bb80 10 client.0 _ll_register_callbacks cb 0x7fc95c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:47:33.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc971f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc97179a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8948cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc971f9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46528/0 (socket says 172.21.15.145:46528) 2023-08-25T04:47:33.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc971f9b700 1 -- 172.21.15.145:0/3664526343 learned_addr learned my addr 172.21.15.145:0/3664526343 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:47:33.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.765+0000 7fc97279c700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce89a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 -- 172.21.15.145:0/3664526343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 msgr2=0x559ce89a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce89a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 -- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 msgr2=0x559ce8948770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:47:33.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 -- 172.21.15.145:0/3664526343 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc968017070 con 0x559ce8941380 2023-08-25T04:47:33.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc971f9b700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:47:33.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc97179a700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8948cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559ce893c330 tx=0x7fc95c024490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5059222b56417c17 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:47:33.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc9557fa700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:47:33.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d070 con 0x559ce8941380 2023-08-25T04:47:33.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce892e8b0 con 0x559ce8941380 2023-08-25T04:47:33.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.766+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fc95c000e40 con 0x559ce8941380 2023-08-25T04:47:33.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c00f8b0 con 0x559ce8941380 2023-08-25T04:47:33.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 10 client.25840 fetch_fsmap learned FSMap version 27 2023-08-25T04:47:33.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ce86b9de0 con 0x559ce8941380 2023-08-25T04:47:33.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ce892e8b0 con 0x559ce8941380 2023-08-25T04:47:33.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc95c02e070 con 0x559ce8941380 2023-08-25T04:47:33.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 10 client.25840 fetch_fsmap finished waiting for FSMap version 27 2023-08-25T04:47:33.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:47:33.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 20 client.25840 populate_metadata read hostname 'smithi145' 2023-08-25T04:47:33.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89a9d90 0x559ce89ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:47:33.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 4 client.25840 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:47:33.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ce8886970 con 0x559ce89a9d90 2023-08-25T04:47:33.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.767+0000 7fc971f9b700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89a9d90 0x559ce89ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:47:33.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc9557fa700 10 client.25840 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:33.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc971f9b700 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89a9d90 0x559ce89ac1b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1e017baf705965 server_cookie=bdb8b159da2721d0 in_seq=0 out_seq=0 2023-08-25T04:47:33.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc9557fa700 1 -- 172.21.15.145:0/3664526343 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559ce8886970 con 0x559ce89a9d90 2023-08-25T04:47:33.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc9557fa700 10 client.25840 handle_command_reply: tid=0 2023-08-25T04:47:33.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 client.25840 shutdown 2023-08-25T04:47:33.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 msgr2=0x559ce8948cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8948cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559ce893c330 tx=0x7fc95c024490 comp rx=0 tx=0).stop 2023-08-25T04:47:33.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 shutdown_connections 2023-08-25T04:47:33.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8941ca0 0x559ce89a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559ce893bc40 0x559ce8948770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559ce89a9d90 0x559ce89ac1b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 --2- 172.21.15.145:0/3664526343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ce8941380 0x559ce8948cb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 >> 172.21.15.145:0/3664526343 conn(0x559ce88b00e0 msgr2=0x559ce89360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.768+0000 7fc9557fa700 0 client.25840 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:47:33.785 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 shutdown_connections 2023-08-25T04:47:33.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc987b5bb80 1 -- 172.21.15.145:0/3664526343 wait complete. 2023-08-25T04:47:33.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc987b5bb80 20 client.25840 trim_cache size 0 max 16384 2023-08-25T04:47:33.788 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:47:33.788 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:47:33.788 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:47:33.788 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:47:33.789 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:47:33.790 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:47:33.791 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:47:33.791 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:47:33.791 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:47:33.791 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:47:33.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 msgr2=0x7fc9580a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 0x7fc9580a2600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc968009be0 tx=0x7fc968009260 comp rx=0 tx=0).stop 2023-08-25T04:47:33.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 msgr2=0x7fc96c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:47:33.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc96400ef30 tx=0x7fc96400c3a0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 shutdown_connections 2023-08-25T04:47:33.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c106880 0x7fc96c1989f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc9580a0140 0x7fc9580a2600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc96c100870 0x7fc96c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.769+0000 7fc9741ff700 1 --2- 172.21.15.145:0/1998061465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc96c101230 0x7fc96c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:47:33.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.770+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 >> 172.21.15.145:0/1998061465 conn(0x7fc96c0fc3a0 msgr2=0x7fc96c10bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:47:33.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.770+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 shutdown_connections 2023-08-25T04:47:33.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:47:33.770+0000 7fc9741ff700 1 -- 172.21.15.145:0/1998061465 wait complete. 2023-08-25T04:47:33.797 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:33 smithi145 ceph-mon[97681]: pgmap v77: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:33.798 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:47:33.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:33 smithi183 ceph-mon[131414]: pgmap v77: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:33 smithi162 ceph-mon[130941]: pgmap v77: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:35 smithi183 ceph-mon[131414]: pgmap v78: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:35.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:35 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:47:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:35 smithi145 ceph-mon[97681]: pgmap v78: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:36.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:35 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:47:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:35 smithi162 ceph-mon[130941]: pgmap v78: 129 pgs: 129 active+clean; 8.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 82 MiB/s wr, 175 op/s 2023-08-25T04:47:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:35 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:47:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:37 smithi145 ceph-mon[97681]: pgmap v79: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 82 MiB/s wr, 354 op/s 2023-08-25T04:47:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:37 smithi162 ceph-mon[130941]: pgmap v79: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 82 MiB/s wr, 354 op/s 2023-08-25T04:47:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:37 smithi183 ceph-mon[131414]: pgmap v79: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 82 MiB/s wr, 354 op/s 2023-08-25T04:47:40.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:39 smithi145 ceph-mon[97681]: pgmap v80: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 308 op/s 2023-08-25T04:47:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:39 smithi162 ceph-mon[130941]: pgmap v80: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 308 op/s 2023-08-25T04:47:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:39 smithi183 ceph-mon[131414]: pgmap v80: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 308 op/s 2023-08-25T04:47:42.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:41 smithi145 ceph-mon[97681]: pgmap v81: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 313 op/s 2023-08-25T04:47:42.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:41 smithi162 ceph-mon[130941]: pgmap v81: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 313 op/s 2023-08-25T04:47:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:41 smithi183 ceph-mon[131414]: pgmap v81: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 63 MiB/s wr, 313 op/s 2023-08-25T04:47:44.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:43 smithi145 ceph-mon[97681]: pgmap v82: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:43 smithi162 ceph-mon[130941]: pgmap v82: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:43 smithi183 ceph-mon[131414]: pgmap v82: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:46.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:45 smithi145 ceph-mon[97681]: pgmap v83: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:45 smithi162 ceph-mon[130941]: pgmap v83: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:45 smithi183 ceph-mon[131414]: pgmap v83: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 KiB/s wr, 183 op/s 2023-08-25T04:47:48.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:47 smithi145 ceph-mon[97681]: pgmap v84: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 306 KiB/s wr, 200 op/s 2023-08-25T04:47:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:47 smithi162 ceph-mon[130941]: pgmap v84: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 306 KiB/s wr, 200 op/s 2023-08-25T04:47:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:47 smithi183 ceph-mon[131414]: pgmap v84: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 306 KiB/s wr, 200 op/s 2023-08-25T04:47:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:49 smithi145 ceph-mon[97681]: pgmap v85: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 287 KiB/s wr, 22 op/s 2023-08-25T04:47:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:49 smithi162 ceph-mon[130941]: pgmap v85: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 287 KiB/s wr, 22 op/s 2023-08-25T04:47:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:49 smithi183 ceph-mon[131414]: pgmap v85: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 287 KiB/s wr, 22 op/s 2023-08-25T04:47:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:51 smithi145 ceph-mon[97681]: pgmap v86: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:47:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:51 smithi162 ceph-mon[130941]: pgmap v86: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:47:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:51 smithi183 ceph-mon[131414]: pgmap v86: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:47:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:53 smithi145 ceph-mon[97681]: pgmap v87: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 354 KiB/s wr, 27 op/s 2023-08-25T04:47:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:53 smithi162 ceph-mon[130941]: pgmap v87: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 354 KiB/s wr, 27 op/s 2023-08-25T04:47:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:53 smithi183 ceph-mon[131414]: pgmap v87: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 354 KiB/s wr, 27 op/s 2023-08-25T04:47:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:55 smithi145 ceph-mon[97681]: pgmap v88: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 355 KiB/s wr, 27 op/s 2023-08-25T04:47:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:55 smithi162 ceph-mon[130941]: pgmap v88: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 355 KiB/s wr, 27 op/s 2023-08-25T04:47:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:55 smithi183 ceph-mon[131414]: pgmap v88: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 355 KiB/s wr, 27 op/s 2023-08-25T04:47:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:57 smithi145 ceph-mon[97681]: pgmap v89: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:47:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:57 smithi162 ceph-mon[130941]: pgmap v89: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:47:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:57 smithi183 ceph-mon[131414]: pgmap v89: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 362 KiB/s wr, 33 op/s 2023-08-25T04:48:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:47:59 smithi145 ceph-mon[97681]: pgmap v90: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-08-25T04:48:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:47:59 smithi162 ceph-mon[130941]: pgmap v90: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-08-25T04:48:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:47:59 smithi183 ceph-mon[131414]: pgmap v90: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-08-25T04:48:02.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:01 smithi145 ceph-mon[97681]: pgmap v91: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 55 op/s 2023-08-25T04:48:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:01 smithi162 ceph-mon[130941]: pgmap v91: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 55 op/s 2023-08-25T04:48:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:01 smithi183 ceph-mon[131414]: pgmap v91: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 55 op/s 2023-08-25T04:48:03.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:48:04.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:03 smithi145 ceph-mon[97681]: pgmap v92: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:04.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.078+0000 7f15565ff700 1 -- 172.21.15.145:0/2681537725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550102360 msgr2=0x7f155010a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.078+0000 7f15565ff700 1 --2- 172.21.15.145:0/2681537725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550102360 0x7f155010a840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1544009990 tx=0x7f1544009ca0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 -- 172.21.15.145:0/2681537725 shutdown_connections 2023-08-25T04:48:04.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 --2- 172.21.15.145:0/2681537725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 0x7f155010d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 --2- 172.21.15.145:0/2681537725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550101a40 0x7f1550101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 --2- 172.21.15.145:0/2681537725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550102360 0x7f155010a840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 -- 172.21.15.145:0/2681537725 >> 172.21.15.145:0/2681537725 conn(0x7f15500fb260 msgr2=0x7f15500fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 -- 172.21.15.145:0/2681537725 shutdown_connections 2023-08-25T04:48:04.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.079+0000 7f15565ff700 1 -- 172.21.15.145:0/2681537725 wait complete. 2023-08-25T04:48:04.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.080+0000 7f15565ff700 1 Processor -- start 2023-08-25T04:48:04.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.080+0000 7f15565ff700 1 -- start start 2023-08-25T04:48:04.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550101a40 0x7f155019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 0x7f1550196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550196ec0 con 0x7f1550101a40 2023-08-25T04:48:04.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550197030 con 0x7f155010ae60 2023-08-25T04:48:04.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f15565ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15501971f0 con 0x7f1550102360 2023-08-25T04:48:04.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33304/0 (socket says 172.21.15.145:33304) 2023-08-25T04:48:04.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f154f7fe700 1 -- 172.21.15.145:0/3508331572 learned_addr learned my addr 172.21.15.145:0/3508331572 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:04.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f154ffff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550101a40 0x7f155019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.081+0000 7f1554b9c700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 0x7f1550196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 -- 172.21.15.145:0/3508331572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 msgr2=0x7f1550196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 0x7f1550196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 -- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550101a40 msgr2=0x7f155019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550101a40 0x7f155019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 -- 172.21.15.145:0/3508331572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15440095f0 con 0x7f1550102360 2023-08-25T04:48:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154f7fe700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1544005910 tx=0x7f1544005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154d7fa700 1 -- 172.21.15.145:0/3508331572 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154401d070 con 0x7f1550102360 2023-08-25T04:48:04.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1550197470 con 0x7f1550102360 2023-08-25T04:48:04.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154d7fa700 1 -- 172.21.15.145:0/3508331572 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1544022470 con 0x7f1550102360 2023-08-25T04:48:04.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f154d7fa700 1 -- 172.21.15.145:0/3508331572 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544017650 con 0x7f1550102360 2023-08-25T04:48:04.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.082+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15501a37d0 con 0x7f1550102360 2023-08-25T04:48:04.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.085+0000 7f154d7fa700 1 -- 172.21.15.145:0/3508331572 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f15440225e0 con 0x7f1550102360 2023-08-25T04:48:04.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 Processor -- start 2023-08-25T04:48:04.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f154d7fa700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 0x7f15380a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 -- start start 2023-08-25T04:48:04.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f154d7fa700 1 -- 172.21.15.145:0/3508331572 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f15440ccba0 con 0x7f1550102360 2023-08-25T04:48:04.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1554b9c700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 0x7f15380a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420bdff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420bdffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd5e7b0 con 0x56420bdfad00 2023-08-25T04:48:04.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd4f780 con 0x56420bdff8d0 2023-08-25T04:48:04.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.087+0000 7f1569eaab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd4d0a0 con 0x56420bdfb620 2023-08-25T04:48:04.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f1554b9c700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 0x7f15380a2580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f155019d4e0 tx=0x7f154800b040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52612/0 (socket says 172.21.15.145:52612) 2023-08-25T04:48:04.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 -- 172.21.15.145:0/1546885565 learned_addr learned my addr 172.21.15.145:0/1546885565 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:04.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154f7fe700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420bdff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f1554b9c700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420bdffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 -- 172.21.15.145:0/1546885565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 msgr2=0x56420bdff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420bdff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 -- 172.21.15.145:0/1546885565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 msgr2=0x56420bdffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420bdffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 -- 172.21.15.145:0/1546885565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56420bbc8630 con 0x56420bdfad00 2023-08-25T04:48:04.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154f7fe700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420bdff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:48:04.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.088+0000 7f154ffff700 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1540009d80 tx=0x7f1540011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8414ecfee38a670 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f15367fc700 1 -- 172.21.15.145:0/1546885565 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540004230 con 0x56420bdfad00 2023-08-25T04:48:04.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f15367fc700 1 -- 172.21.15.145:0/1546885565 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1540004390 con 0x56420bdfad00 2023-08-25T04:48:04.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f15367fc700 1 -- 172.21.15.145:0/1546885565 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540019860 con 0x56420bdfad00 2023-08-25T04:48:04.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 -- 172.21.15.145:0/1546885565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 msgr2=0x56420bdfb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1540009d80 tx=0x7f1540011720 comp rx=0 tx=0).stop 2023-08-25T04:48:04.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 -- 172.21.15.145:0/1546885565 shutdown_connections 2023-08-25T04:48:04.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420bdffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420bdfb0e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1546885565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420bdff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 -- 172.21.15.145:0/1546885565 >> 172.21.15.145:0/1546885565 conn(0x56420bd6b0e0 msgr2=0x56420bdf1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 -- 172.21.15.145:0/1546885565 shutdown_connections 2023-08-25T04:48:04.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.089+0000 7f1569eaab80 1 -- 172.21.15.145:0/1546885565 wait complete. 2023-08-25T04:48:04.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.090+0000 7f1569eaab80 1 -- start start 2023-08-25T04:48:04.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.090+0000 7f1569eaab80 1 Processor -- start 2023-08-25T04:48:04.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420be04310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be04a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd48990 con 0x56420bdfad00 2023-08-25T04:48:04.105 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bce6bb0 con 0x56420bdff8d0 2023-08-25T04:48:04.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bce68b0 con 0x56420bdfb620 2023-08-25T04:48:04.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1569eaab80 10 client.0 _ll_register_callbacks cb 0x56420bc30a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:48:04.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52614/0 (socket says 172.21.15.145:52614) 2023-08-25T04:48:04.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 -- 172.21.15.145:0/2543947346 learned_addr learned my addr 172.21.15.145:0/2543947346 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:04.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1554b9c700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be04a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154f7fe700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420be04310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 -- 172.21.15.145:0/2543947346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 msgr2=0x56420be04310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420be04310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 -- 172.21.15.145:0/2543947346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 msgr2=0x56420be04a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be04a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154ffff700 1 -- 172.21.15.145:0/2543947346 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15501a37d0 con 0x56420bdfad00 2023-08-25T04:48:04.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f1554b9c700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be04a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:48:04.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.091+0000 7f154f7fe700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420be04310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:48:04.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f154ffff700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1540011a70 tx=0x7f1540003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e98b068fa6c6fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:48:04.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540005d60 con 0x56420bdfad00 2023-08-25T04:48:04.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56420bdc61c0 con 0x56420bdfad00 2023-08-25T04:48:04.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1540004350 con 0x56420bdfad00 2023-08-25T04:48:04.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1540004510 con 0x56420bdfad00 2023-08-25T04:48:04.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f1569eaab80 10 client.15831 fetch_fsmap learned FSMap version 28 2023-08-25T04:48:04.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f15400099e0 con 0x56420bdfad00 2023-08-25T04:48:04.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56420bdc61c0 con 0x56420bdfad00 2023-08-25T04:48:04.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.092+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1540021040 con 0x56420bdfad00 2023-08-25T04:48:04.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T04:48:04.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:48:04.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 20 client.15831 populate_metadata read hostname 'smithi145' 2023-08-25T04:48:04.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be14ab0 0x56420be16ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 4 client.15831 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:48:04.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.093+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56420bceeba0 con 0x56420be14ab0 2023-08-25T04:48:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:03 smithi162 ceph-mon[130941]: pgmap v92: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:03 smithi183 ceph-mon[131414]: pgmap v92: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:04.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.198+0000 7f154f7fe700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be14ab0 0x56420be16ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.199+0000 7f154f7fe700 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be14ab0 0x56420be16ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355df41701110db0 server_cookie=c24643ee9d160635 in_seq=0 out_seq=0 2023-08-25T04:48:04.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.199+0000 7f152f7fe700 10 client.15831 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:04.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.203+0000 7f152f7fe700 1 -- 172.21.15.145:0/2543947346 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56420bceeba0 con 0x56420be14ab0 2023-08-25T04:48:04.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.203+0000 7f152f7fe700 10 client.15831 handle_command_reply: tid=0 2023-08-25T04:48:04.206 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.203+0000 7f1569eaab80 1 client.15831 shutdown 2023-08-25T04:48:04.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.211+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 msgr2=0x56420be07100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.211+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1540011a70 tx=0x7f1540003d00 comp rx=0 tx=0).stop 2023-08-25T04:48:04.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 shutdown_connections 2023-08-25T04:48:04.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be04a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfad00 0x56420be07100 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be14ab0 0x56420be16ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 --2- 172.21.15.145:0/2543947346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfb620 0x56420be04310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 >> 172.21.15.145:0/2543947346 conn(0x56420bd6b0e0 msgr2=0x56420bdeb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.212+0000 7f152f7fe700 0 client.15831 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.213+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 shutdown_connections 2023-08-25T04:48:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.213+0000 7f1569eaab80 1 -- 172.21.15.145:0/2543947346 wait complete. 2023-08-25T04:48:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.214+0000 7f1569eaab80 20 client.15831 trim_cache size 0 max 16384 2023-08-25T04:48:04.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 Processor -- start 2023-08-25T04:48:04.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 -- start start 2023-08-25T04:48:04.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be06c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be03c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bbca920 con 0x56420bdfb620 2023-08-25T04:48:04.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bdfba00 con 0x56420bdff8d0 2023-08-25T04:48:04.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.243+0000 7f1569eaab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420be121f0 con 0x56420bdfad00 2023-08-25T04:48:04.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be06c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be0bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33324/0 (socket says 172.21.15.145:33324) 2023-08-25T04:48:04.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154ffff700 1 -- 172.21.15.145:0/999603079 learned_addr learned my addr 172.21.15.145:0/999603079 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:04.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f1554b9c700 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be03c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 -- 172.21.15.145:0/999603079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 msgr2=0x56420be0bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be0bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 -- 172.21.15.145:0/999603079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 msgr2=0x56420be03c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be03c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 -- 172.21.15.145:0/999603079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56420bb74de0 con 0x56420bdfb620 2023-08-25T04:48:04.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.244+0000 7f154f7fe700 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be06c30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f15440c9fb0 tx=0x7f15440b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be54864cc52b09ae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.245+0000 7f1536ffd700 1 -- 172.21.15.145:0/999603079 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544099960 con 0x56420bdfb620 2023-08-25T04:48:04.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.245+0000 7f1536ffd700 1 -- 172.21.15.145:0/999603079 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154409a700 con 0x56420bdfb620 2023-08-25T04:48:04.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.245+0000 7f1536ffd700 1 -- 172.21.15.145:0/999603079 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440228c0 con 0x56420bdfb620 2023-08-25T04:48:04.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.245+0000 7f1569eaab80 1 -- 172.21.15.145:0/999603079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 msgr2=0x56420be06c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.245+0000 7f1569eaab80 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be06c30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f15440c9fb0 tx=0x7f15440b9040 comp rx=0 tx=0).stop 2023-08-25T04:48:04.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 -- 172.21.15.145:0/999603079 shutdown_connections 2023-08-25T04:48:04.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be03c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 --2- 172.21.15.145:0/999603079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be06c30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 -- 172.21.15.145:0/999603079 >> 172.21.15.145:0/999603079 conn(0x56420bd6b0e0 msgr2=0x56420be01cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 -- 172.21.15.145:0/999603079 shutdown_connections 2023-08-25T04:48:04.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 -- 172.21.15.145:0/999603079 wait complete. 2023-08-25T04:48:04.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 -- start start 2023-08-25T04:48:04.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.246+0000 7f1569eaab80 1 Processor -- start 2023-08-25T04:48:04.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be0a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be0a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420be679f0 con 0x56420bdfb620 2023-08-25T04:48:04.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420be67b60 con 0x56420bdff8d0 2023-08-25T04:48:04.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420be67d20 con 0x56420bdfad00 2023-08-25T04:48:04.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1569eaab80 10 client.0 _ll_register_callbacks cb 0x7f1548009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:48:04.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f154ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be0a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f154ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33328/0 (socket says 172.21.15.145:33328) 2023-08-25T04:48:04.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f154ffff700 1 -- 172.21.15.145:0/1615064621 learned_addr learned my addr 172.21.15.145:0/1615064621 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:04.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.247+0000 7f1554b9c700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be0a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 -- 172.21.15.145:0/1615064621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 msgr2=0x56420be0a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be0a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 -- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 msgr2=0x56420be0a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:04.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be0a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 -- 172.21.15.145:0/1615064621 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1550197470 con 0x56420bdfad00 2023-08-25T04:48:04.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f154ffff700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15400095c0 tx=0x7f1540005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7397374ccb042212 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f152f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:48:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540009cb0 con 0x56420bdfad00 2023-08-25T04:48:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56420bde98b0 con 0x56420bdfad00 2023-08-25T04:48:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1540003d70 con 0x56420bdfad00 2023-08-25T04:48:04.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1540019460 con 0x56420bdfad00 2023-08-25T04:48:04.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f1569eaab80 10 client.25858 fetch_fsmap learned FSMap version 28 2023-08-25T04:48:04.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.248+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56420bb5dc60 con 0x56420bdfad00 2023-08-25T04:48:04.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56420bde98b0 con 0x56420bdfad00 2023-08-25T04:48:04.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1540021070 con 0x56420bdfad00 2023-08-25T04:48:04.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 10 client.25858 fetch_fsmap finished waiting for FSMap version 28 2023-08-25T04:48:04.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:48:04.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 20 client.25858 populate_metadata read hostname 'smithi145' 2023-08-25T04:48:04.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be69120 0x56420be6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:04.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 4 client.25858 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:48:04.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56420bd41970 con 0x56420be69120 2023-08-25T04:48:04.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.249+0000 7f154f7fe700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be69120 0x56420be6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:04.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f152f7fe700 10 client.25858 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:04.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f154f7fe700 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be69120 0x56420be6b540 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fc97eab0c498f8 server_cookie=d9132d258fa8ac7e in_seq=0 out_seq=0 2023-08-25T04:48:04.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f152f7fe700 1 -- 172.21.15.145:0/1615064621 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56420bd41970 con 0x56420be69120 2023-08-25T04:48:04.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f152f7fe700 10 client.25858 handle_command_reply: tid=0 2023-08-25T04:48:04.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 client.25858 shutdown 2023-08-25T04:48:04.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 msgr2=0x56420be09bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f15400095c0 tx=0x7f1540005d70 comp rx=0 tx=0).stop 2023-08-25T04:48:04.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 shutdown_connections 2023-08-25T04:48:04.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56420bdff8d0 0x56420be0a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.274 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56420be69120 0x56420be6b540 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56420bdfad00 0x56420be09bb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f152f7fe700 0 client.25858 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:04.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 --2- 172.21.15.145:0/1615064621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56420bdfb620 0x56420be0a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 >> 172.21.15.145:0/1615064621 conn(0x56420bd6b0e0 msgr2=0x56420bdeb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.250+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 shutdown_connections 2023-08-25T04:48:04.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f1569eaab80 1 -- 172.21.15.145:0/1615064621 wait complete. 2023-08-25T04:48:04.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f1569eaab80 20 client.25858 trim_cache size 0 max 16384 2023-08-25T04:48:04.276 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:48:04.277 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:48:04.277 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:48:04.277 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:48:04.278 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:48:04.278 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:48:04.278 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:48:04.278 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:48:04.279 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:48:04.279 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:48:04.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 msgr2=0x7f15380a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 0x7f15380a2580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f155019d4e0 tx=0x7f154800b040 comp rx=0 tx=0).stop 2023-08-25T04:48:04.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 msgr2=0x7f155019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:04.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1544005910 tx=0x7f1544005740 comp rx=0 tx=0).stop 2023-08-25T04:48:04.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 shutdown_connections 2023-08-25T04:48:04.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f15380a00c0 0x7f15380a2580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155010ae60 0x7f1550196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1550101a40 0x7f155019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 --2- 172.21.15.145:0/3508331572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1550102360 0x7f155019cd20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:04.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.251+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 >> 172.21.15.145:0/3508331572 conn(0x7f15500fb260 msgr2=0x7f15500fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:04.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.252+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 shutdown_connections 2023-08-25T04:48:04.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:04.252+0000 7f15565ff700 1 -- 172.21.15.145:0/3508331572 wait complete. 2023-08-25T04:48:04.287 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:48:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:05 smithi145 ceph-mon[97681]: pgmap v93: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:05 smithi162 ceph-mon[130941]: pgmap v93: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:05 smithi183 ceph-mon[131414]: pgmap v93: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 44 op/s 2023-08-25T04:48:08.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:07 smithi145 ceph-mon[97681]: pgmap v94: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 50 op/s 2023-08-25T04:48:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:07 smithi162 ceph-mon[130941]: pgmap v94: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 50 op/s 2023-08-25T04:48:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:07 smithi183 ceph-mon[131414]: pgmap v94: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 50 op/s 2023-08-25T04:48:10.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:09 smithi145 ceph-mon[97681]: pgmap v95: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 45 op/s 2023-08-25T04:48:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:09 smithi162 ceph-mon[130941]: pgmap v95: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 45 op/s 2023-08-25T04:48:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:09 smithi183 ceph-mon[131414]: pgmap v95: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 45 op/s 2023-08-25T04:48:12.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:11 smithi145 ceph-mon[97681]: pgmap v96: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 49 op/s 2023-08-25T04:48:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:11 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:48:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:11 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:11 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:48:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:11 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:11 smithi162 ceph-mon[130941]: pgmap v96: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 49 op/s 2023-08-25T04:48:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:11 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:48:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:11 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:11 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:48:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:11 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:11 smithi183 ceph-mon[131414]: pgmap v96: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 49 op/s 2023-08-25T04:48:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:11 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:48:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:11 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:11 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:48:12.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:11 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:14.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:13 smithi145 ceph-mon[97681]: pgmap v97: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:13 smithi162 ceph-mon[130941]: pgmap v97: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:13 smithi183 ceph-mon[131414]: pgmap v97: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:15 smithi145 ceph-mon[97681]: pgmap v98: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:15 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:48:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:15 smithi162 ceph-mon[130941]: pgmap v98: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:15 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:48:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:15 smithi183 ceph-mon[131414]: pgmap v98: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 9 op/s 2023-08-25T04:48:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:15 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:48:17.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:16 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:48:17.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:16 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:48:17.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:16 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:48:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:17 smithi145 ceph-mon[97681]: pgmap v99: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 15 MiB/s wr, 89 op/s 2023-08-25T04:48:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:17 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:48:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:17 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:48:18.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:17 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:48:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:17 smithi162 ceph-mon[130941]: pgmap v99: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 15 MiB/s wr, 89 op/s 2023-08-25T04:48:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:17 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:48:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:17 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:48:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:17 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:48:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:17 smithi183 ceph-mon[131414]: pgmap v99: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 15 MiB/s wr, 89 op/s 2023-08-25T04:48:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:17 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:48:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:17 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:48:18.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:17 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:48:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:19 smithi145 ceph-mon[97681]: pgmap v100: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:19 smithi162 ceph-mon[130941]: pgmap v100: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:19 smithi183 ceph-mon[131414]: pgmap v100: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:22.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:21 smithi145 ceph-mon[97681]: pgmap v101: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 MiB/s wr, 86 op/s 2023-08-25T04:48:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:21 smithi145 ceph-mon[97681]: scrub summary: active paths [/] 2023-08-25T04:48:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:21 smithi145 ceph-mon[97681]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:21 smithi162 ceph-mon[130941]: pgmap v101: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 MiB/s wr, 86 op/s 2023-08-25T04:48:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:21 smithi162 ceph-mon[130941]: scrub summary: active paths [/] 2023-08-25T04:48:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:21 smithi162 ceph-mon[130941]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:21 smithi183 ceph-mon[131414]: pgmap v101: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 MiB/s wr, 86 op/s 2023-08-25T04:48:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:21 smithi183 ceph-mon[131414]: scrub summary: active paths [/] 2023-08-25T04:48:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:21 smithi183 ceph-mon[131414]: scrub summary: idle+waiting paths [/] 2023-08-25T04:48:24.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:23 smithi145 ceph-mon[97681]: pgmap v102: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:23 smithi162 ceph-mon[130941]: pgmap v102: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:23 smithi183 ceph-mon[131414]: pgmap v102: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:26.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:25 smithi145 ceph-mon[97681]: pgmap v103: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:25 smithi162 ceph-mon[130941]: pgmap v103: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:25 smithi183 ceph-mon[131414]: pgmap v103: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 83 op/s 2023-08-25T04:48:28.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:27 smithi145 ceph-mon[97681]: pgmap v104: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 89 op/s 2023-08-25T04:48:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:27 smithi162 ceph-mon[130941]: pgmap v104: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 89 op/s 2023-08-25T04:48:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:27 smithi183 ceph-mon[131414]: pgmap v104: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 89 op/s 2023-08-25T04:48:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:29 smithi145 ceph-mon[97681]: pgmap v105: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-08-25T04:48:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:29 smithi162 ceph-mon[130941]: pgmap v105: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-08-25T04:48:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:29 smithi183 ceph-mon[131414]: pgmap v105: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-08-25T04:48:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:31 smithi145 ceph-mon[97681]: pgmap v106: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-08-25T04:48:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:31 smithi162 ceph-mon[130941]: pgmap v106: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-08-25T04:48:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:31 smithi183 ceph-mon[131414]: pgmap v106: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-08-25T04:48:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:33 smithi145 ceph-mon[97681]: pgmap v107: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:33 smithi162 ceph-mon[130941]: pgmap v107: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:33 smithi183 ceph-mon[131414]: pgmap v107: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:34.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:48:34.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 -- 172.21.15.145:0/2596122416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0100890 msgr2=0x7fd9a0100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/2596122416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0100890 0x7fd9a0100d10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd994009990 tx=0x7fd994009ca0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 -- 172.21.15.145:0/2596122416 shutdown_connections 2023-08-25T04:48:34.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/2596122416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a0101250 0x7fd9a010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/2596122416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a01068a0 0x7fd9a0106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/2596122416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0100890 0x7fd9a0100d10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 -- 172.21.15.145:0/2596122416 >> 172.21.15.145:0/2596122416 conn(0x7fd9a00fc380 msgr2=0x7fd9a00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.531+0000 7fd9a65ff700 1 -- 172.21.15.145:0/2596122416 shutdown_connections 2023-08-25T04:48:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.532+0000 7fd9a65ff700 1 -- 172.21.15.145:0/2596122416 wait complete. 2023-08-25T04:48:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.533+0000 7fd9a65ff700 1 Processor -- start 2023-08-25T04:48:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.533+0000 7fd9a65ff700 1 -- start start 2023-08-25T04:48:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.533+0000 7fd9a65ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a65ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0101250 0x7fd9a0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a65ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 0x7fd9a01a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a65ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0073990 con 0x7fd9a0101250 2023-08-25T04:48:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a65ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01a50a0 con 0x7fd9a0100890 2023-08-25T04:48:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a65ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01a51e0 con 0x7fd9a01068a0 2023-08-25T04:48:34.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46654/0 (socket says 172.21.15.145:46654) 2023-08-25T04:48:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 -- 172.21.15.145:0/617980393 learned_addr learned my addr 172.21.15.145:0/617980393 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0101250 0x7fd9a0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 0x7fd9a01a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 -- 172.21.15.145:0/617980393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 msgr2=0x7fd9a01a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 0x7fd9a01a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 -- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0101250 msgr2=0x7fd9a0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0101250 0x7fd9a0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd99ffff700 1 -- 172.21.15.145:0/617980393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940095f0 con 0x7fd9a0100890 2023-08-25T04:48:34.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.534+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 0x7fd9a01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:48:34.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd99ffff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd99000e9a0 tx=0x7fd99000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd99d7fa700 1 -- 172.21.15.145:0/617980393 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990009790 con 0x7fd9a0100890 2023-08-25T04:48:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a01a5440 con 0x7fd9a0100890 2023-08-25T04:48:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd99d7fa700 1 -- 172.21.15.145:0/617980393 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd990010430 con 0x7fd9a0100890 2023-08-25T04:48:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd99d7fa700 1 -- 172.21.15.145:0/617980393 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990018680 con 0x7fd9a0100890 2023-08-25T04:48:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.535+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a01a5970 con 0x7fd9a0100890 2023-08-25T04:48:34.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.539+0000 7fd99d7fa700 1 -- 172.21.15.145:0/617980393 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fd9900105a0 con 0x7fd9a0100890 2023-08-25T04:48:34.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.540+0000 7fd9b9ffbb80 1 Processor -- start 2023-08-25T04:48:34.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.540+0000 7fd9b9ffbb80 1 -- start start 2023-08-25T04:48:34.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7107b0 con 0x55e65d7aeca0 2023-08-25T04:48:34.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d701780 con 0x55e65d7a8c40 2023-08-25T04:48:34.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d6ff0a0 con 0x55e65d7ae380 2023-08-25T04:48:34.559 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd99d7fa700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 0x7fd9880a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33916/0 (socket says 172.21.15.145:33916) 2023-08-25T04:48:34.560 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 -- 172.21.15.145:0/2705044596 learned_addr learned my addr 172.21.15.145:0/2705044596 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:34.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd99ffff700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.561 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd99d7fa700 1 -- 172.21.15.145:0/617980393 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fd990014070 con 0x7fd9a0100890 2023-08-25T04:48:34.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 -- 172.21.15.145:0/2705044596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 msgr2=0x55e65d7a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.562 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 -- 172.21.15.145:0/2705044596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 msgr2=0x55e65d7a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.563 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd9a4b9c700 1 -- 172.21.15.145:0/2705044596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65d57a630 con 0x55e65d7ae380 2023-08-25T04:48:34.564 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.541+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:48:34.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.542+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 0x7fd9880a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.542+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd998003ef0 tx=0x7fd998004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b332d15ccdd5e0e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.565 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.542+0000 7fd9867fc700 1 -- 172.21.15.145:0/2705044596 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980090b0 con 0x55e65d7ae380 2023-08-25T04:48:34.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.542+0000 7fd9867fc700 1 -- 172.21.15.145:0/2705044596 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd99800f040 con 0x55e65d7ae380 2023-08-25T04:48:34.566 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.542+0000 7fd9867fc700 1 -- 172.21.15.145:0/2705044596 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980188a0 con 0x55e65d7ae380 2023-08-25T04:48:34.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 0x7fd9880a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd998022000 tx=0x7fd99801f000 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2705044596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 msgr2=0x55e65d7ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.567 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd998003ef0 tx=0x7fd998004320 comp rx=0 tx=0).stop 2023-08-25T04:48:34.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2705044596 shutdown_connections 2023-08-25T04:48:34.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7ae760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.568 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2705044596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2705044596 >> 172.21.15.145:0/2705044596 conn(0x55e65d71d0e0 msgr2=0x55e65d7a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2705044596 shutdown_connections 2023-08-25T04:48:34.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.543+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2705044596 wait complete. 2023-08-25T04:48:34.569 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.544+0000 7fd9b9ffbb80 1 -- start start 2023-08-25T04:48:34.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.544+0000 7fd9b9ffbb80 1 Processor -- start 2023-08-25T04:48:34.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.570 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d6fa990 con 0x55e65d7aeca0 2023-08-25T04:48:34.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d698bb0 con 0x55e65d7a8c40 2023-08-25T04:48:34.571 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d6988b0 con 0x55e65d7ae380 2023-08-25T04:48:34.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9b9ffbb80 10 client.0 _ll_register_callbacks cb 0x55e65d5e2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:48:34.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.572 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33926/0 (socket says 172.21.15.145:33926) 2023-08-25T04:48:34.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 -- 172.21.15.145:0/1244728727 learned_addr learned my addr 172.21.15.145:0/1244728727 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:34.573 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 -- 172.21.15.145:0/1244728727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 msgr2=0x55e65d7b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.574 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 -- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 msgr2=0x55e65d7b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.575 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.545+0000 7fd99f7fe700 1 -- 172.21.15.145:0/1244728727 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd998003bd0 con 0x55e65d7ae380 2023-08-25T04:48:34.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd994015000 tx=0x7fd99400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb7daafb80fc28f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd97f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:48:34.576 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99401d070 con 0x55e65d7ae380 2023-08-25T04:48:34.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65d7781c0 con 0x55e65d7ae380 2023-08-25T04:48:34.577 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fd994004500 con 0x55e65d7ae380 2023-08-25T04:48:34.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd99402d740 con 0x55e65d7ae380 2023-08-25T04:48:34.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd9b9ffbb80 10 client.25870 fetch_fsmap learned FSMap version 29 2023-08-25T04:48:34.578 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e65d526de0 con 0x55e65d7ae380 2023-08-25T04:48:34.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.546+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e65d7781c0 con 0x55e65d7ae380 2023-08-25T04:48:34.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd9940046c0 con 0x55e65d7ae380 2023-08-25T04:48:34.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 10 client.25870 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:48:34.579 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:48:34.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 20 client.25870 populate_metadata read hostname 'smithi145' 2023-08-25T04:48:34.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d7c2880 0x55e65d7c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 4 client.25870 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:48:34.580 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e65d6a0ba0 con 0x55e65d7c2880 2023-08-25T04:48:34.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.547+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d7c2880 0x55e65d7c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.548+0000 7fd97f7fe700 10 client.25870 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:34.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.548+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d7c2880 0x55e65d7c4ca0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8efcbc6aab3eaa server_cookie=764681f5c4e434ad in_seq=0 out_seq=0 2023-08-25T04:48:34.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.548+0000 7fd97f7fe700 1 -- 172.21.15.145:0/1244728727 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e65d6a0ba0 con 0x55e65d7c2880 2023-08-25T04:48:34.581 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.548+0000 7fd97f7fe700 10 client.25870 handle_command_reply: tid=0 2023-08-25T04:48:34.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.548+0000 7fd9b9ffbb80 1 client.25870 shutdown 2023-08-25T04:48:34.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 msgr2=0x55e65d7b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.582 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd994015000 tx=0x7fd99400bf60 comp rx=0 tx=0).stop 2023-08-25T04:48:34.583 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 shutdown_connections 2023-08-25T04:48:34.586 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7a8c40 0x55e65d7b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d7c2880 0x55e65d7c4ca0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1244728727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b24d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 >> 172.21.15.145:0/1244728727 conn(0x55e65d71d0e0 msgr2=0x55e65d79db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd97f7fe700 0 client.25870 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 shutdown_connections 2023-08-25T04:48:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1244728727 wait complete. 2023-08-25T04:48:34.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.549+0000 7fd9b9ffbb80 20 client.25870 trim_cache size 0 max 16384 2023-08-25T04:48:34.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.573+0000 7fd9b9ffbb80 1 Processor -- start 2023-08-25T04:48:34.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.573+0000 7fd9b9ffbb80 1 -- start start 2023-08-25T04:48:34.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7ae380 0x55e65d7ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d57c920 con 0x55e65d7aeca0 2023-08-25T04:48:34.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7a4d40 con 0x55e65d7ae380 2023-08-25T04:48:34.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7a4180 con 0x55e65d7a8c40 2023-08-25T04:48:34.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33938/0 (socket says 172.21.15.145:33938) 2023-08-25T04:48:34.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 -- 172.21.15.145:0/1022564556 learned_addr learned my addr 172.21.15.145:0/1022564556 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:34.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 -- 172.21.15.145:0/1022564556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7ae380 msgr2=0x55e65d7ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7ae380 0x55e65d7ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 -- 172.21.15.145:0/1022564556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 msgr2=0x55e65d7ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd99ffff700 1 -- 172.21.15.145:0/1022564556 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65d50fc60 con 0x55e65d7a8c40 2023-08-25T04:48:34.602 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.574+0000 7fd9a4b9c700 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:48:34.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.575+0000 7fd99ffff700 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c2d0 tx=0x7fd99001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83582cd4166fc38d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.575+0000 7fd986ffd700 1 -- 172.21.15.145:0/1022564556 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990099ea0 con 0x55e65d7a8c40 2023-08-25T04:48:34.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.575+0000 7fd986ffd700 1 -- 172.21.15.145:0/1022564556 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd990098e90 con 0x55e65d7a8c40 2023-08-25T04:48:34.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.575+0000 7fd986ffd700 1 -- 172.21.15.145:0/1022564556 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b8320 con 0x55e65d7a8c40 2023-08-25T04:48:34.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1022564556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 msgr2=0x55e65d7b97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c2d0 tx=0x7fd99001c910 comp rx=0 tx=0).stop 2023-08-25T04:48:34.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1022564556 shutdown_connections 2023-08-25T04:48:34.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7aeca0 0x55e65d7ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7a8c40 0x55e65d7b97a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/1022564556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7ae380 0x55e65d7ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1022564556 >> 172.21.15.145:0/1022564556 conn(0x55e65d71d0e0 msgr2=0x55e65d7bdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1022564556 shutdown_connections 2023-08-25T04:48:34.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/1022564556 wait complete. 2023-08-25T04:48:34.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 -- start start 2023-08-25T04:48:34.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.576+0000 7fd9b9ffbb80 1 Processor -- start 2023-08-25T04:48:34.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7a8c40 0x55e65d7b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7aeca0 0x55e65d7b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7b9b80 con 0x55e65d7a8c40 2023-08-25T04:48:34.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7b8230 con 0x55e65d7aeca0 2023-08-25T04:48:34.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65d7b83f0 con 0x55e65d7ae380 2023-08-25T04:48:34.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd9b9ffbb80 10 client.0 _ll_register_callbacks cb 0x7fd994009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:48:34.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:33942/0 (socket says 172.21.15.145:33942) 2023-08-25T04:48:34.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 -- 172.21.15.145:0/2101810776 learned_addr learned my addr 172.21.15.145:0/2101810776 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:48:34.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7a8c40 0x55e65d7b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 -- 172.21.15.145:0/2101810776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7aeca0 msgr2=0x55e65d7b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7aeca0 0x55e65d7b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 -- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7a8c40 msgr2=0x55e65d7b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:48:34.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7a8c40 0x55e65d7b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.577+0000 7fd99f7fe700 1 -- 172.21.15.145:0/2101810776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a01a5970 con 0x55e65d7ae380 2023-08-25T04:48:34.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd99f7fe700 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b6f0 tx=0x7fd994005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fae21324352d23f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:48:34.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd97f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:48:34.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99401d070 con 0x55e65d7ae380 2023-08-25T04:48:34.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65d79b8b0 con 0x55e65d7ae380 2023-08-25T04:48:34.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fd994000e40 con 0x55e65d7ae380 2023-08-25T04:48:34.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd994005bc0 con 0x55e65d7ae380 2023-08-25T04:48:34.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd9b9ffbb80 10 client.25876 fetch_fsmap learned FSMap version 29 2023-08-25T04:48:34.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e65d54d250 con 0x55e65d7ae380 2023-08-25T04:48:34.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.578+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e65d79b8b0 con 0x55e65d7ae380 2023-08-25T04:48:34.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd994024a20 con 0x55e65d7ae380 2023-08-25T04:48:34.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 10 client.25876 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:48:34.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:48:34.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 20 client.25876 populate_metadata read hostname 'smithi145' 2023-08-25T04:48:34.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d8176e0 0x55e65d819b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:48:34.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 4 client.25876 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:48:34.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e65d6f3970 con 0x55e65d8176e0 2023-08-25T04:48:34.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.579+0000 7fd99ffff700 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d8176e0 0x55e65d819b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:48:34.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd97f7fe700 10 client.25876 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:34.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd99ffff700 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d8176e0 0x55e65d819b00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee1ddc9a1c9f2ba server_cookie=53193ba1eed07eaa in_seq=0 out_seq=0 2023-08-25T04:48:34.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd97f7fe700 1 -- 172.21.15.145:0/2101810776 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e65d6f3970 con 0x55e65d8176e0 2023-08-25T04:48:34.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd97f7fe700 10 client.25876 handle_command_reply: tid=0 2023-08-25T04:48:34.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 client.25876 shutdown 2023-08-25T04:48:34.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 msgr2=0x55e65d7b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b6f0 tx=0x7fd994005910 comp rx=0 tx=0).stop 2023-08-25T04:48:34.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 shutdown_connections 2023-08-25T04:48:34.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e65d7aeca0 0x55e65d7b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e65d7a8c40 0x55e65d7b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55e65d8176e0 0x55e65d819b00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 --2- 172.21.15.145:0/2101810776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e65d7ae380 0x55e65d7b7780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd97f7fe700 0 client.25876 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:48:34.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 >> 172.21.15.145:0/2101810776 conn(0x55e65d71d0e0 msgr2=0x55e65d7ac0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 shutdown_connections 2023-08-25T04:48:34.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.580+0000 7fd9b9ffbb80 1 -- 172.21.15.145:0/2101810776 wait complete. 2023-08-25T04:48:34.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9b9ffbb80 20 client.25876 trim_cache size 0 max 16384 2023-08-25T04:48:34.620 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:48:34.621 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:48:34.621 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:48:34.621 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:48:34.621 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:48:34.622 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:48:34.622 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:48:34.622 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:48:34.622 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:48:34.622 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:48:34.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 msgr2=0x7fd9880a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 0x7fd9880a2580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd998022000 tx=0x7fd99801f000 comp rx=0 tx=0).stop 2023-08-25T04:48:34.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 msgr2=0x7fd9a0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:48:34.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd99000e9a0 tx=0x7fd99000ecb0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 shutdown_connections 2023-08-25T04:48:34.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a01068a0 0x7fd9a01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd9880a00c0 0x7fd9880a2580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9a0100890 0x7fd9a0072c20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 --2- 172.21.15.145:0/617980393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a0101250 0x7fd9a0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:48:34.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 >> 172.21.15.145:0/617980393 conn(0x7fd9a00fc380 msgr2=0x7fd9a00fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:48:34.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 shutdown_connections 2023-08-25T04:48:34.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:48:34.581+0000 7fd9a65ff700 1 -- 172.21.15.145:0/617980393 wait complete. 2023-08-25T04:48:34.627 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:48:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:35 smithi145 ceph-mon[97681]: pgmap v108: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:35 smithi162 ceph-mon[130941]: pgmap v108: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:35 smithi183 ceph-mon[131414]: pgmap v108: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-08-25T04:48:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:37 smithi145 ceph-mon[97681]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-08-25T04:48:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:37 smithi162 ceph-mon[130941]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-08-25T04:48:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:37 smithi183 ceph-mon[131414]: pgmap v109: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-08-25T04:48:40.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:39 smithi145 ceph-mon[97681]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 7 op/s 2023-08-25T04:48:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:39 smithi162 ceph-mon[130941]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 7 op/s 2023-08-25T04:48:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:39 smithi183 ceph-mon[131414]: pgmap v110: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 7 op/s 2023-08-25T04:48:42.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:41 smithi145 ceph-mon[97681]: pgmap v111: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-08-25T04:48:42.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:41 smithi162 ceph-mon[130941]: pgmap v111: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-08-25T04:48:42.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:41 smithi183 ceph-mon[131414]: pgmap v111: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-08-25T04:48:44.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:43 smithi145 ceph-mon[97681]: pgmap v112: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:43 smithi162 ceph-mon[130941]: pgmap v112: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:43 smithi183 ceph-mon[131414]: pgmap v112: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:46.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:45 smithi145 ceph-mon[97681]: pgmap v113: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:45 smithi162 ceph-mon[130941]: pgmap v113: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:45 smithi183 ceph-mon[131414]: pgmap v113: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-08-25T04:48:48.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:47 smithi145 ceph-mon[97681]: pgmap v114: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-08-25T04:48:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:47 smithi162 ceph-mon[130941]: pgmap v114: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-08-25T04:48:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:47 smithi183 ceph-mon[131414]: pgmap v114: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-08-25T04:48:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:49 smithi145 ceph-mon[97681]: pgmap v115: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-08-25T04:48:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:49 smithi162 ceph-mon[130941]: pgmap v115: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-08-25T04:48:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:49 smithi183 ceph-mon[131414]: pgmap v115: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-08-25T04:48:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:51 smithi145 ceph-mon[97681]: pgmap v116: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-08-25T04:48:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:51 smithi162 ceph-mon[130941]: pgmap v116: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-08-25T04:48:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:51 smithi183 ceph-mon[131414]: pgmap v116: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-08-25T04:48:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:53 smithi145 ceph-mon[97681]: pgmap v117: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:53 smithi162 ceph-mon[130941]: pgmap v117: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:53 smithi183 ceph-mon[131414]: pgmap v117: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:55 smithi145 ceph-mon[97681]: pgmap v118: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:55 smithi162 ceph-mon[130941]: pgmap v118: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:55 smithi183 ceph-mon[131414]: pgmap v118: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-08-25T04:48:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:57 smithi145 ceph-mon[97681]: pgmap v119: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-08-25T04:48:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:57 smithi162 ceph-mon[130941]: pgmap v119: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-08-25T04:48:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:57 smithi183 ceph-mon[131414]: pgmap v119: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-08-25T04:49:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:48:59 smithi145 ceph-mon[97681]: pgmap v120: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2023-08-25T04:49:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:48:59 smithi162 ceph-mon[130941]: pgmap v120: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2023-08-25T04:49:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:48:59 smithi183 ceph-mon[131414]: pgmap v120: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2023-08-25T04:49:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:01 smithi162 ceph-mon[130941]: pgmap v121: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-08-25T04:49:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:01 smithi183 ceph-mon[131414]: pgmap v121: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-08-25T04:49:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:01 smithi145 ceph-mon[97681]: pgmap v121: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-08-25T04:49:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:03 smithi162 ceph-mon[130941]: pgmap v122: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:03 smithi183 ceph-mon[131414]: pgmap v122: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:03 smithi145 ceph-mon[97681]: pgmap v122: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:04.629 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:49:04.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 -- 172.21.15.145:0/1366347333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788101a30 msgr2=0x7f4788101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 --2- 172.21.15.145:0/1366347333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788101a30 0x7f4788101e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4778009990 tx=0x7f4778009ca0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 -- 172.21.15.145:0/1366347333 shutdown_connections 2023-08-25T04:49:04.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 --2- 172.21.15.145:0/1366347333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 0x7f478810d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 --2- 172.21.15.145:0/1366347333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788101a30 0x7f4788101e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 --2- 172.21.15.145:0/1366347333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788102350 0x7f478810a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 -- 172.21.15.145:0/1366347333 >> 172.21.15.145:0/1366347333 conn(0x7f47880fb270 msgr2=0x7f47880fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 -- 172.21.15.145:0/1366347333 shutdown_connections 2023-08-25T04:49:04.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.865+0000 7f478d3ff700 1 -- 172.21.15.145:0/1366347333 wait complete. 2023-08-25T04:49:04.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.866+0000 7f478d3ff700 1 Processor -- start 2023-08-25T04:49:04.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 -- start start 2023-08-25T04:49:04.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788102350 0x7f478819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 0x7f4788198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881990b0 con 0x7f4788102350 2023-08-25T04:49:04.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788199220 con 0x7f478810ae50 2023-08-25T04:49:04.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f478d3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881993e0 con 0x7f4788101a30 2023-08-25T04:49:04.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48228/0 (socket says 172.21.15.145:48228) 2023-08-25T04:49:04.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f47867fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788102350 0x7f478819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.867+0000 7f4786ffd700 1 -- 172.21.15.145:0/2203713437 learned_addr learned my addr 172.21.15.145:0/2203713437 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:04.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f47877fe700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 0x7f4788198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 -- 172.21.15.145:0/2203713437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 msgr2=0x7f4788198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 0x7f4788198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 -- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788102350 msgr2=0x7f478819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788102350 0x7f478819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 -- 172.21.15.145:0/2203713437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47780095f0 con 0x7f4788101a30 2023-08-25T04:49:04.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4778005910 tx=0x7f4778004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f476ffff700 1 -- 172.21.15.145:0/2203713437 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477801d070 con 0x7f4788101a30 2023-08-25T04:49:04.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f476ffff700 1 -- 172.21.15.145:0/2203713437 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4778022470 con 0x7f4788101a30 2023-08-25T04:49:04.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788199660 con 0x7f4788101a30 2023-08-25T04:49:04.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.868+0000 7f476ffff700 1 -- 172.21.15.145:0/2203713437 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778017650 con 0x7f4788101a30 2023-08-25T04:49:04.875 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.869+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47881a59c0 con 0x7f4788101a30 2023-08-25T04:49:04.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.872+0000 7f476ffff700 1 -- 172.21.15.145:0/2203713437 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f477800fad0 con 0x7f4788101a30 2023-08-25T04:49:04.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.873+0000 7f47a0cdbb80 1 Processor -- start 2023-08-25T04:49:04.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.873+0000 7f47a0cdbb80 1 -- start start 2023-08-25T04:49:04.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa697c7b0 con 0x558fa6a14c40 2023-08-25T04:49:04.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa696d780 con 0x558fa6a1aca0 2023-08-25T04:49:04.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa696b0a0 con 0x558fa6a1a380 2023-08-25T04:49:04.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47877fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f476ffff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 0x7f47700a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47877fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48230/0 (socket says 172.21.15.145:48230) 2023-08-25T04:49:04.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47877fe700 1 -- 172.21.15.145:0/3061516307 learned_addr learned my addr 172.21.15.145:0/3061516307 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:04.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f4786ffd700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f47867fc700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.880 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.874+0000 7f476ffff700 1 -- 172.21.15.145:0/2203713437 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f47780cce30 con 0x7f4788101a30 2023-08-25T04:49:04.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 -- 172.21.15.145:0/3061516307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 msgr2=0x558fa6a14700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a14700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 -- 172.21.15.145:0/3061516307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 msgr2=0x558fa6a150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.881 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 -- 172.21.15.145:0/3061516307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa67e6630 con 0x558fa6a1a380 2023-08-25T04:49:04.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47867fc700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:49:04.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 0x7f47700a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f47877fe700 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4780009e10 tx=0x7f47800077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc978fe8b2afa16 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f476d7fa700 1 -- 172.21.15.145:0/3061516307 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780018d90 con 0x558fa6a1a380 2023-08-25T04:49:04.882 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f476d7fa700 1 -- 172.21.15.145:0/3061516307 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4780023900 con 0x558fa6a1a380 2023-08-25T04:49:04.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.875+0000 7f476d7fa700 1 -- 172.21.15.145:0/3061516307 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47800175f0 con 0x558fa6a1a380 2023-08-25T04:49:04.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47877fe700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 0x7f47700a2600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4780020000 tx=0x7f478001d000 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/3061516307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 msgr2=0x558fa6a1a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4780009e10 tx=0x7f47800077e0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.883 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/3061516307 shutdown_connections 2023-08-25T04:49:04.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a1a760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/3061516307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a14700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/3061516307 >> 172.21.15.145:0/3061516307 conn(0x558fa69890e0 msgr2=0x558fa6a0f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/3061516307 shutdown_connections 2023-08-25T04:49:04.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.876+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/3061516307 wait complete. 2023-08-25T04:49:04.885 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.877+0000 7f47a0cdbb80 1 -- start start 2023-08-25T04:49:04.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.877+0000 7f47a0cdbb80 1 Processor -- start 2023-08-25T04:49:04.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1a380 0x558fa6a1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6966990 con 0x558fa6a1a380 2023-08-25T04:49:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6904bb0 con 0x558fa6a1aca0 2023-08-25T04:49:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa69048b0 con 0x558fa6a14c40 2023-08-25T04:49:04.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47a0cdbb80 10 client.0 _ll_register_callbacks cb 0x558fa684ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:49:04.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48246/0 (socket says 172.21.15.145:48246) 2023-08-25T04:49:04.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 -- 172.21.15.145:0/2230336268 learned_addr learned my addr 172.21.15.145:0/2230336268 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:04.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47867fc700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1a380 0x558fa6a1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f47877fe700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a1ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 -- 172.21.15.145:0/2230336268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 msgr2=0x558fa6a1ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a1ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 -- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1a380 msgr2=0x558fa6a1e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.891 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.878+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1a380 0x558fa6a1e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f4786ffd700 1 -- 172.21.15.145:0/2230336268 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780009a40 con 0x558fa6a14c40 2023-08-25T04:49:04.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f47877fe700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a1ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:49:04.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4778029000 tx=0x7f477806d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea68f643788a0f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.892 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f476aff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:49:04.893 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778099d10 con 0x558fa6a14c40 2023-08-25T04:49:04.893 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f4778066470 con 0x558fa6a14c40 2023-08-25T04:49:04.893 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fa69e41c0 con 0x558fa6a14c40 2023-08-25T04:49:04.893 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477809aab0 con 0x558fa6a14c40 2023-08-25T04:49:04.893 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f47a0cdbb80 10 client.25888 fetch_fsmap learned FSMap version 29 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558fa6792de0 con 0x558fa6a14c40 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.879+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fa69e41c0 con 0x558fa6a14c40 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f477809a610 con 0x558fa6a14c40 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 10 client.25888 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:49:04.894 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 20 client.25888 populate_metadata read hostname 'smithi145' 2023-08-25T04:49:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a2e300 0x558fa6a30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 4 client.25888 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:49:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fa690cba0 con 0x558fa6a2e300 2023-08-25T04:49:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.880+0000 7f47867fc700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a2e300 0x558fa6a30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.881+0000 7f47867fc700 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a2e300 0x558fa6a30720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0ecb934275c0cf server_cookie=457be906fe113ee5 in_seq=0 out_seq=0 2023-08-25T04:49:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.881+0000 7f476aff5700 10 client.25888 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.881+0000 7f476aff5700 1 -- 172.21.15.145:0/2230336268 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fa690cba0 con 0x558fa6a2e300 2023-08-25T04:49:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.881+0000 7f476aff5700 10 client.25888 handle_command_reply: tid=0 2023-08-25T04:49:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.881+0000 7f47a0cdbb80 1 client.25888 shutdown 2023-08-25T04:49:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 msgr2=0x558fa6a1dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4778029000 tx=0x7f477806d5f0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 shutdown_connections 2023-08-25T04:49:04.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a1ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a2e300 0x558fa6a30720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.897 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1dfb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2230336268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1a380 0x558fa6a1e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f476aff5700 0 client.25888 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 >> 172.21.15.145:0/2230336268 conn(0x558fa69890e0 msgr2=0x558fa6a09b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 shutdown_connections 2023-08-25T04:49:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2230336268 wait complete. 2023-08-25T04:49:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.882+0000 7f47a0cdbb80 20 client.25888 trim_cache size 0 max 16384 2023-08-25T04:49:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.902+0000 7f47a0cdbb80 1 Processor -- start 2023-08-25T04:49:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.902+0000 7f47a0cdbb80 1 -- start start 2023-08-25T04:49:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 0x558fa6a190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1aca0 0x558fa6a195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa67e8920 con 0x558fa6a1aca0 2023-08-25T04:49:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6a0ba20 con 0x558fa6a1a380 2023-08-25T04:49:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6a111f0 con 0x558fa6a14c40 2023-08-25T04:49:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.906 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48252/0 (socket says 172.21.15.145:48252) 2023-08-25T04:49:04.906 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 -- 172.21.15.145:0/1877324481 learned_addr learned my addr 172.21.15.145:0/1877324481 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:04.906 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47877fe700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1aca0 0x558fa6a195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.906 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f47867fc700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 0x558fa6a190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 -- 172.21.15.145:0/1877324481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 msgr2=0x558fa6a190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 0x558fa6a190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 -- 172.21.15.145:0/1877324481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1aca0 msgr2=0x558fa6a195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1aca0 0x558fa6a195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.903+0000 7f4786ffd700 1 -- 172.21.15.145:0/1877324481 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa677bc60 con 0x558fa6a14c40 2023-08-25T04:49:04.907 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.904+0000 7f47867fc700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 0x558fa6a190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:49:04.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.904+0000 7f4786ffd700 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f47780b1040 tx=0x7f4778000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e57e1c37e225a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.904+0000 7f476dffb700 1 -- 172.21.15.145:0/1877324481 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778099d10 con 0x558fa6a14c40 2023-08-25T04:49:04.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.904+0000 7f476dffb700 1 -- 172.21.15.145:0/1877324481 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47780226d0 con 0x558fa6a14c40 2023-08-25T04:49:04.911 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.904+0000 7f476dffb700 1 -- 172.21.15.145:0/1877324481 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780663b0 con 0x558fa6a14c40 2023-08-25T04:49:04.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/1877324481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 msgr2=0x558fa6a1e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f47780b1040 tx=0x7f4778000bc0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/1877324481 shutdown_connections 2023-08-25T04:49:04.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a1aca0 0x558fa6a195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a14c40 0x558fa6a1e860 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/1877324481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1a380 0x558fa6a190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/1877324481 >> 172.21.15.145:0/1877324481 conn(0x558fa69890e0 msgr2=0x558fa6a18130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/1877324481 shutdown_connections 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/1877324481 wait complete. 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.905+0000 7f47a0cdbb80 1 -- start start 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 Processor -- start 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a23970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6a7b600 con 0x558fa6a14c40 2023-08-25T04:49:04.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6a7b770 con 0x558fa6a1aca0 2023-08-25T04:49:04.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa6a7b930 con 0x558fa6a1a380 2023-08-25T04:49:04.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47a0cdbb80 10 client.0 _ll_register_callbacks cb 0x7f47780a00b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:49:04.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.906+0000 7f47867fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:48260/0 (socket says 172.21.15.145:48260) 2023-08-25T04:49:04.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 -- 172.21.15.145:0/2236370748 learned_addr learned my addr 172.21.15.145:0/2236370748 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:04.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a23970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47877fe700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 -- 172.21.15.145:0/2236370748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 msgr2=0x558fa6a7b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a7b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 -- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 msgr2=0x558fa6a23970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:04.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a23970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 -- 172.21.15.145:0/2236370748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fa6792de0 con 0x558fa6a1a380 2023-08-25T04:49:04.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a23970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:49:04.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47877fe700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:49:04.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f47867fc700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f477c009ce0 tx=0x7f477c0043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b361266890e2fe1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:04.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f476aff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:49:04.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.907+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c017030 con 0x558fa6a1a380 2023-08-25T04:49:04.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fa6a078b0 con 0x558fa6a1a380 2023-08-25T04:49:04.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f477c003cc0 con 0x558fa6a1a380 2023-08-25T04:49:04.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477c003e80 con 0x558fa6a1a380 2023-08-25T04:49:04.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 10 client.25894 fetch_fsmap learned FSMap version 29 2023-08-25T04:49:04.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558fa67b9250 con 0x558fa6a1a380 2023-08-25T04:49:04.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fa6a078b0 con 0x558fa6a1a380 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f477c010aa0 con 0x558fa6a1a380 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 10 client.25894 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 20 client.25894 populate_metadata read hostname 'smithi145' 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a83170 0x558fa6a85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 4 client.25894 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:49:04.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.908+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fa695f970 con 0x558fa6a83170 2023-08-25T04:49:04.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a83170 0x558fa6a85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:04.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f4786ffd700 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a83170 0x558fa6a85590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e80c9c1587bdba server_cookie=908a385f02cf4844 in_seq=0 out_seq=0 2023-08-25T04:49:04.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f476aff5700 10 client.25894 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:04.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f476aff5700 1 -- 172.21.15.145:0/2236370748 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558fa695f970 con 0x558fa6a83170 2023-08-25T04:49:04.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f476aff5700 10 client.25894 handle_command_reply: tid=0 2023-08-25T04:49:04.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f47a0cdbb80 1 client.25894 shutdown 2023-08-25T04:49:04.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 msgr2=0x558fa6a23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.909+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f477c009ce0 tx=0x7f477c0043d0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 shutdown_connections 2023-08-25T04:49:04.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558fa6a1aca0 0x558fa6a7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558fa6a14c40 0x558fa6a23970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558fa6a83170 0x558fa6a85590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 --2- 172.21.15.145:0/2236370748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fa6a1a380 0x558fa6a23eb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.923 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 >> 172.21.15.145:0/2236370748 conn(0x558fa69890e0 msgr2=0x558fa6a190b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f476aff5700 0 client.25894 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:04.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 shutdown_connections 2023-08-25T04:49:04.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 1 -- 172.21.15.145:0/2236370748 wait complete. 2023-08-25T04:49:04.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.910+0000 7f47a0cdbb80 20 client.25894 trim_cache size 0 max 16384 2023-08-25T04:49:04.925 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:49:04.925 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:49:04.925 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:49:04.925 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:49:04.925 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:49:04.926 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:49:04.926 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:49:04.926 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:49:04.926 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:49:04.926 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:49:04.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 msgr2=0x7f47700a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 0x7f47700a2600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4780020000 tx=0x7f478001d000 comp rx=0 tx=0).stop 2023-08-25T04:49:04.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 msgr2=0x7f478819e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:04.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4778005910 tx=0x7f4778004730 comp rx=0 tx=0).stop 2023-08-25T04:49:04.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 shutdown_connections 2023-08-25T04:49:04.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47700a0140 0x7f47700a2600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f478810ae50 0x7f4788198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788101a30 0x7f478819e9d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 --2- 172.21.15.145:0/2203713437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4788102350 0x7f478819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:04.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 >> 172.21.15.145:0/2203713437 conn(0x7f47880fb270 msgr2=0x7f478810d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:04.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 shutdown_connections 2023-08-25T04:49:04.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:04.911+0000 7f478d3ff700 1 -- 172.21.15.145:0/2203713437 wait complete. 2023-08-25T04:49:04.933 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:49:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:05 smithi162 ceph-mon[130941]: pgmap v123: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:05 smithi183 ceph-mon[131414]: pgmap v123: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:05 smithi145 ceph-mon[97681]: pgmap v123: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-08-25T04:49:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:07 smithi162 ceph-mon[130941]: pgmap v124: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-08-25T04:49:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:07 smithi183 ceph-mon[131414]: pgmap v124: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-08-25T04:49:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:07 smithi145 ceph-mon[97681]: pgmap v124: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-08-25T04:49:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:09 smithi162 ceph-mon[130941]: pgmap v125: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:49:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:09 smithi183 ceph-mon[131414]: pgmap v125: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:49:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:09 smithi145 ceph-mon[97681]: pgmap v125: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:49:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:11 smithi162 ceph-mon[130941]: pgmap v126: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:49:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:11 smithi183 ceph-mon[131414]: pgmap v126: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:49:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:11 smithi145 ceph-mon[97681]: pgmap v126: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:49:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:13 smithi162 ceph-mon[130941]: pgmap v127: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:13 smithi183 ceph-mon[131414]: pgmap v127: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:13 smithi145 ceph-mon[97681]: pgmap v127: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:15 smithi162 ceph-mon[130941]: pgmap v128: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:15 smithi183 ceph-mon[131414]: pgmap v128: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:15 smithi145 ceph-mon[97681]: pgmap v128: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2023-08-25T04:49:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:17 smithi162 ceph-mon[130941]: pgmap v129: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T04:49:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:17 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:49:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:17 smithi183 ceph-mon[131414]: pgmap v129: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T04:49:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:17 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:49:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:17 smithi145 ceph-mon[97681]: pgmap v129: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-08-25T04:49:18.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:17 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:49:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:18 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:49:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:18 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:49:19.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:18 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:49:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:18 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:49:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:18 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:49:19.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:18 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:49:19.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:18 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:49:19.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:18 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:49:19.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:18 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:49:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:19 smithi162 ceph-mon[130941]: pgmap v130: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:49:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:19 smithi183 ceph-mon[131414]: pgmap v130: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:49:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:19 smithi145 ceph-mon[97681]: pgmap v130: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:49:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:21 smithi162 ceph-mon[130941]: pgmap v131: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:49:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:21 smithi183 ceph-mon[131414]: pgmap v131: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:49:22.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:21 smithi145 ceph-mon[97681]: pgmap v131: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:49:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:23 smithi162 ceph-mon[130941]: pgmap v132: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:23 smithi183 ceph-mon[131414]: pgmap v132: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:23 smithi145 ceph-mon[97681]: pgmap v132: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:25 smithi162 ceph-mon[130941]: pgmap v133: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:25 smithi183 ceph-mon[131414]: pgmap v133: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:25 smithi145 ceph-mon[97681]: pgmap v133: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:27 smithi162 ceph-mon[130941]: pgmap v134: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:27 smithi183 ceph-mon[131414]: pgmap v134: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:27 smithi145 ceph-mon[97681]: pgmap v134: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:29 smithi162 ceph-mon[130941]: pgmap v135: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:29 smithi183 ceph-mon[131414]: pgmap v135: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:30.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:29 smithi145 ceph-mon[97681]: pgmap v135: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:31 smithi162 ceph-mon[130941]: pgmap v136: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:31 smithi183 ceph-mon[131414]: pgmap v136: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:31 smithi145 ceph-mon[97681]: pgmap v136: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:33 smithi162 ceph-mon[130941]: pgmap v137: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:33 smithi183 ceph-mon[131414]: pgmap v137: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:33 smithi145 ceph-mon[97681]: pgmap v137: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:34.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:49:35.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 -- 172.21.15.145:0/1841463334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 msgr2=0x7f021c10a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 --2- 172.21.15.145:0/1841463334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c10a830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0210009990 tx=0x7f0210009ca0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 -- 172.21.15.145:0/1841463334 shutdown_connections 2023-08-25T04:49:35.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 --2- 172.21.15.145:0/1841463334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 0x7f021c10d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 --2- 172.21.15.145:0/1841463334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 0x7f021c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 --2- 172.21.15.145:0/1841463334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c10a830 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.158+0000 7f02209ff700 1 -- 172.21.15.145:0/1841463334 >> 172.21.15.145:0/1841463334 conn(0x7f021c0fb270 msgr2=0x7f021c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.159+0000 7f02209ff700 1 -- 172.21.15.145:0/1841463334 shutdown_connections 2023-08-25T04:49:35.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.159+0000 7f02209ff700 1 -- 172.21.15.145:0/1841463334 wait complete. 2023-08-25T04:49:35.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.160+0000 7f02209ff700 1 Processor -- start 2023-08-25T04:49:35.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.160+0000 7f02209ff700 1 -- start start 2023-08-25T04:49:35.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 0x7f021c0ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 0x7f021c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1003f0 con 0x7f021c10ae50 2023-08-25T04:49:35.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1a5180 con 0x7f021c101a30 2023-08-25T04:49:35.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f02209ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1a52c0 con 0x7f021c102350 2023-08-25T04:49:35.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41244/0 (socket says 172.21.15.145:41244) 2023-08-25T04:49:35.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 -- 172.21.15.145:0/2762092271 learned_addr learned my addr 172.21.15.145:0/2762092271 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:35.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f021ad9d700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 0x7f021c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f021a59c700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 0x7f021c0ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 -- 172.21.15.145:0/2762092271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 msgr2=0x7f021c0ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 0x7f021c0ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 -- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 msgr2=0x7f021c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 0x7f021c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.161+0000 7f0219d9b700 1 -- 172.21.15.145:0/2762092271 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100095f0 con 0x7f021c102350 2023-08-25T04:49:35.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f0219d9b700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0210005910 tx=0x7f0210004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f020b7fe700 1 -- 172.21.15.145:0/2762092271 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021001d070 con 0x7f021c102350 2023-08-25T04:49:35.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f020b7fe700 1 -- 172.21.15.145:0/2762092271 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0210022470 con 0x7f021c102350 2023-08-25T04:49:35.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c1a54c0 con 0x7f021c102350 2023-08-25T04:49:35.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f020b7fe700 1 -- 172.21.15.145:0/2762092271 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210017650 con 0x7f021c102350 2023-08-25T04:49:35.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.162+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c1a5990 con 0x7f021c102350 2023-08-25T04:49:35.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.166+0000 7f020b7fe700 1 -- 172.21.15.145:0/2762092271 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f02100225e0 con 0x7f021c102350 2023-08-25T04:49:35.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.167+0000 7f020b7fe700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 0x7f02040a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f020b7fe700 1 -- 172.21.15.145:0/2762092271 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f02100cca30 con 0x7f021c102350 2023-08-25T04:49:35.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f0234390b80 1 Processor -- start 2023-08-25T04:49:35.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f021a59c700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 0x7f02040a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f0234390b80 1 -- start start 2023-08-25T04:49:35.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f0234390b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 0x55c8b806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.168+0000 7f0234390b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806ea20 0x55c8b8072790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f0234390b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 0x55c8b8073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f0234390b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7fcd7b0 con 0x55c8b806e100 2023-08-25T04:49:35.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f0234390b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7fbe780 con 0x55c8b8072cd0 2023-08-25T04:49:35.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f0234390b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7fbc0a0 con 0x55c8b806ea20 2023-08-25T04:49:35.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021a59c700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 0x7f02040a2600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f020c009c60 tx=0x7f020c009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 0x55c8b806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 0x55c8b806e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38362/0 (socket says 172.21.15.145:38362) 2023-08-25T04:49:35.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021a59c700 1 -- 172.21.15.145:0/3074018585 learned_addr learned my addr 172.21.15.145:0/3074018585 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:35.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 0x55c8b8073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f0219d9b700 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806ea20 0x55c8b8072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 -- 172.21.15.145:0/3074018585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806ea20 msgr2=0x55c8b8072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806ea20 0x55c8b8072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 -- 172.21.15.145:0/3074018585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 msgr2=0x55c8b806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 0x55c8b806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.169+0000 7f021ad9d700 1 -- 172.21.15.145:0/3074018585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b7e37630 con 0x55c8b8072cd0 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.170+0000 7f021ad9d700 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 0x55c8b8073130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0214011550 tx=0x7f0214009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237594f46b369233 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.170+0000 7f0208ff9700 1 -- 172.21.15.145:0/3074018585 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400e040 con 0x55c8b8072cd0 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.170+0000 7f0208ff9700 1 -- 172.21.15.145:0/3074018585 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021400f040 con 0x55c8b8072cd0 2023-08-25T04:49:35.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.170+0000 7f0208ff9700 1 -- 172.21.15.145:0/3074018585 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004430 con 0x55c8b8072cd0 2023-08-25T04:49:35.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- 172.21.15.145:0/3074018585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 msgr2=0x55c8b8073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 0x55c8b8073130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0214011550 tx=0x7f0214009920 comp rx=0 tx=0).stop 2023-08-25T04:49:35.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- 172.21.15.145:0/3074018585 shutdown_connections 2023-08-25T04:49:35.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b8072cd0 0x55c8b8073130 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806e100 0x55c8b806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 --2- 172.21.15.145:0/3074018585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806ea20 0x55c8b8072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- 172.21.15.145:0/3074018585 >> 172.21.15.145:0/3074018585 conn(0x55c8b7fda0e0 msgr2=0x55c8b8060410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- 172.21.15.145:0/3074018585 shutdown_connections 2023-08-25T04:49:35.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- 172.21.15.145:0/3074018585 wait complete. 2023-08-25T04:49:35.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.171+0000 7f0234390b80 1 -- start start 2023-08-25T04:49:35.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 Processor -- start 2023-08-25T04:49:35.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b8077700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b8077e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7fb7990 con 0x55c8b8072cd0 2023-08-25T04:49:35.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7f55bb0 con 0x55c8b806ea20 2023-08-25T04:49:35.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.172+0000 7f0234390b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7f558b0 con 0x55c8b806e100 2023-08-25T04:49:35.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f0234390b80 10 client.0 _ll_register_callbacks cb 0x55c8b7e9fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:49:35.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41262/0 (socket says 172.21.15.145:41262) 2023-08-25T04:49:35.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 -- 172.21.15.145:0/39439187 learned_addr learned my addr 172.21.15.145:0/39439187 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:35.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021ad9d700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b8077e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f0219d9b700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b8077700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 -- 172.21.15.145:0/39439187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 msgr2=0x55c8b8077700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b8077700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.183 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 -- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 msgr2=0x55c8b8077e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b8077e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 -- 172.21.15.145:0/39439187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0214011230 con 0x55c8b806e100 2023-08-25T04:49:35.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f0219d9b700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:49:35.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f021a59c700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f020c003900 tx=0x7f020c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c16b12f9b71737 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.173+0000 7f0201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:49:35.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c022070 con 0x55c8b806e100 2023-08-25T04:49:35.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b80351c0 con 0x55c8b806e100 2023-08-25T04:49:35.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f020c01d820 con 0x55c8b806e100 2023-08-25T04:49:35.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f020c01d9e0 con 0x55c8b806e100 2023-08-25T04:49:35.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 10 client.25906 fetch_fsmap learned FSMap version 29 2023-08-25T04:49:35.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8b7de3de0 con 0x55c8b806e100 2023-08-25T04:49:35.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8b80351c0 con 0x55c8b806e100 2023-08-25T04:49:35.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f020c01c410 con 0x55c8b806e100 2023-08-25T04:49:35.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:49:35.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 20 client.25906 populate_metadata read hostname 'smithi145' 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b8088040 0x55c8b808a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 4 client.25906 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.174+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b7f5dba0 con 0x55c8b8088040 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.175+0000 7f0219d9b700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b8088040 0x55c8b808a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.187 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.175+0000 7f0219d9b700 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b8088040 0x55c8b808a460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ad533fddb48ef6 server_cookie=eac8c9409d0af7c in_seq=0 out_seq=0 2023-08-25T04:49:35.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.175+0000 7f0201ffb700 10 client.25906 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:35.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0201ffb700 1 -- 172.21.15.145:0/39439187 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8b7f5dba0 con 0x55c8b8088040 2023-08-25T04:49:35.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0201ffb700 10 client.25906 handle_command_reply: tid=0 2023-08-25T04:49:35.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 client.25906 shutdown 2023-08-25T04:49:35.188 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 msgr2=0x55c8b807a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f020c003900 tx=0x7f020c0047c0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 shutdown_connections 2023-08-25T04:49:35.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b8077e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b8088040 0x55c8b808a460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.189 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807a4f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 --2- 172.21.15.145:0/39439187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0201ffb700 0 client.25906 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:35.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 >> 172.21.15.145:0/39439187 conn(0x55c8b7fda0e0 msgr2=0x55c8b805a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 shutdown_connections 2023-08-25T04:49:35.190 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.176+0000 7f0234390b80 1 -- 172.21.15.145:0/39439187 wait complete. 2023-08-25T04:49:35.191 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.177+0000 7f0234390b80 20 client.25906 trim_cache size 0 max 16384 2023-08-25T04:49:35.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 Processor -- start 2023-08-25T04:49:35.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 -- start start 2023-08-25T04:49:35.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.197 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b80723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b80740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b7e39920 con 0x55c8b8072cd0 2023-08-25T04:49:35.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b806ee30 con 0x55c8b806ea20 2023-08-25T04:49:35.198 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.195+0000 7f0234390b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b806e4e0 con 0x55c8b806e100 2023-08-25T04:49:35.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41264/0 (socket says 172.21.15.145:41264) 2023-08-25T04:49:35.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 -- 172.21.15.145:0/3961333915 learned_addr learned my addr 172.21.15.145:0/3961333915 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:35.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021ad9d700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b80740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.200 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f0219d9b700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b80723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 -- 172.21.15.145:0/3961333915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 msgr2=0x55c8b80723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b80723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 -- 172.21.15.145:0/3961333915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 msgr2=0x55c8b80740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b80740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.201 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 -- 172.21.15.145:0/3961333915 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b808f090 con 0x55c8b806e100 2023-08-25T04:49:35.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f0219d9b700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b80723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:49:35.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.196+0000 7f021a59c700 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f020c005950 tx=0x7f020c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377469d799d09af9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f02097fa700 1 -- 172.21.15.145:0/3961333915 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c022070 con 0x55c8b806e100 2023-08-25T04:49:35.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f02097fa700 1 -- 172.21.15.145:0/3961333915 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020c01cea0 con 0x55c8b806e100 2023-08-25T04:49:35.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f02097fa700 1 -- 172.21.15.145:0/3961333915 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c005040 con 0x55c8b806e100 2023-08-25T04:49:35.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 -- 172.21.15.145:0/3961333915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 msgr2=0x55c8b807f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f020c005950 tx=0x7f020c0092d0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 -- 172.21.15.145:0/3961333915 shutdown_connections 2023-08-25T04:49:35.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b8072cd0 0x55c8b80740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b806e100 0x55c8b807f010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 --2- 172.21.15.145:0/3961333915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806ea20 0x55c8b80723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 -- 172.21.15.145:0/3961333915 >> 172.21.15.145:0/3961333915 conn(0x55c8b7fda0e0 msgr2=0x55c8b8071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 -- 172.21.15.145:0/3961333915 shutdown_connections 2023-08-25T04:49:35.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.197+0000 7f0234390b80 1 -- 172.21.15.145:0/3961333915 wait complete. 2023-08-25T04:49:35.204 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.198+0000 7f0234390b80 1 -- start start 2023-08-25T04:49:35.205 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.198+0000 7f0234390b80 1 Processor -- start 2023-08-25T04:49:35.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 0x55c8b80762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.207 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806ea20 0x55c8b8076810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b80dbfa0 con 0x55c8b806ea20 2023-08-25T04:49:35.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b80dc110 con 0x55c8b806e100 2023-08-25T04:49:35.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b80dc2d0 con 0x55c8b8072cd0 2023-08-25T04:49:35.208 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0234390b80 10 client.0 _ll_register_callbacks cb 0x7f020c004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:49:35.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:41266/0 (socket says 172.21.15.145:41266) 2023-08-25T04:49:35.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 -- 172.21.15.145:0/1798317778 learned_addr learned my addr 172.21.15.145:0/1798317778 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:49:35.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 0x55c8b80762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.209 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f0219d9b700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806ea20 0x55c8b8076810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 -- 172.21.15.145:0/1798317778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 msgr2=0x55c8b80762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 0x55c8b80762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 -- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806ea20 msgr2=0x55c8b8076810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806ea20 0x55c8b8076810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021ad9d700 1 -- 172.21.15.145:0/1798317778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b7de3de0 con 0x55c8b8072cd0 2023-08-25T04:49:35.210 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.199+0000 7f021a59c700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 0x55c8b80762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:49:35.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f021ad9d700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0214011550 tx=0x7f0214009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde92ef278711db6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:49:35.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:49:35.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400e040 con 0x55c8b8072cd0 2023-08-25T04:49:35.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b80588b0 con 0x55c8b8072cd0 2023-08-25T04:49:35.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f0214007910 con 0x55c8b8072cd0 2023-08-25T04:49:35.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0214011e50 con 0x55c8b8072cd0 2023-08-25T04:49:35.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0234390b80 10 client.25912 fetch_fsmap learned FSMap version 29 2023-08-25T04:49:35.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.200+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8b7dccc60 con 0x55c8b8072cd0 2023-08-25T04:49:35.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8b80588b0 con 0x55c8b8072cd0 2023-08-25T04:49:35.212 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f021401b020 con 0x55c8b8072cd0 2023-08-25T04:49:35.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 10 client.25912 fetch_fsmap finished waiting for FSMap version 29 2023-08-25T04:49:35.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:49:35.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 20 client.25912 populate_metadata read hostname 'smithi145' 2023-08-25T04:49:35.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b80dd9e0 0x55c8b80dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:49:35.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 4 client.25912 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:49:35.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8b7fb0970 con 0x55c8b80dd9e0 2023-08-25T04:49:35.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.201+0000 7f021a59c700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b80dd9e0 0x55c8b80dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:49:35.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f021a59c700 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b80dd9e0 0x55c8b80dfe00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4c7d87de599000 server_cookie=6e5b68ac2cfdff29 in_seq=0 out_seq=0 2023-08-25T04:49:35.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0201ffb700 10 client.25912 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:35.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0201ffb700 1 -- 172.21.15.145:0/1798317778 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c8b7fb0970 con 0x55c8b80dd9e0 2023-08-25T04:49:35.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0201ffb700 10 client.25912 handle_command_reply: tid=0 2023-08-25T04:49:35.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 client.25912 shutdown 2023-08-25T04:49:35.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 msgr2=0x55c8b8076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0214011550 tx=0x7f0214009510 comp rx=0 tx=0).stop 2023-08-25T04:49:35.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 shutdown_connections 2023-08-25T04:49:35.216 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55c8b80dd9e0 0x55c8b80dfe00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.216 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8b8072cd0 0x55c8b8076d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.216 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c8b806e100 0x55c8b80762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.216 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 --2- 172.21.15.145:0/1798317778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c8b806ea20 0x55c8b8076810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0201ffb700 0 client.25912 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:49:35.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 >> 172.21.15.145:0/1798317778 conn(0x55c8b7fda0e0 msgr2=0x55c8b8071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 shutdown_connections 2023-08-25T04:49:35.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.202+0000 7f0234390b80 1 -- 172.21.15.145:0/1798317778 wait complete. 2023-08-25T04:49:35.217 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f0234390b80 20 client.25912 trim_cache size 0 max 16384 2023-08-25T04:49:35.218 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:49:35.218 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:49:35.218 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:49:35.219 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:49:35.219 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:49:35.219 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:49:35.219 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:49:35.219 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:49:35.220 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:49:35.220 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:49:35.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 msgr2=0x7f02040a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 0x7f02040a2600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f020c009c60 tx=0x7f020c009260 comp rx=0 tx=0).stop 2023-08-25T04:49:35.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 msgr2=0x7f021c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:49:35.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0210005910 tx=0x7f0210004730 comp rx=0 tx=0).stop 2023-08-25T04:49:35.221 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 shutdown_connections 2023-08-25T04:49:35.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f021c10ae50 0x7f021c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f02040a0140 0x7f02040a2600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f021c101a30 0x7f021c0ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 --2- 172.21.15.145:0/2762092271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f021c102350 0x7f021c0ffbc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:49:35.222 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.203+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 >> 172.21.15.145:0/2762092271 conn(0x7f021c0fb270 msgr2=0x7f021c10d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:49:35.223 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.204+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 shutdown_connections 2023-08-25T04:49:35.223 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:49:35.204+0000 7f02209ff700 1 -- 172.21.15.145:0/2762092271 wait complete. 2023-08-25T04:49:35.224 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:49:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:35 smithi162 ceph-mon[130941]: pgmap v138: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:35 smithi183 ceph-mon[131414]: pgmap v138: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:35 smithi145 ceph-mon[97681]: pgmap v138: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:37 smithi162 ceph-mon[130941]: pgmap v139: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:37 smithi183 ceph-mon[131414]: pgmap v139: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:38.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:37 smithi145 ceph-mon[97681]: pgmap v139: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:39 smithi162 ceph-mon[130941]: pgmap v140: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:39 smithi183 ceph-mon[131414]: pgmap v140: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:39 smithi145 ceph-mon[97681]: pgmap v140: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:41 smithi162 ceph-mon[130941]: pgmap v141: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:41 smithi183 ceph-mon[131414]: pgmap v141: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:42.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:41 smithi145 ceph-mon[97681]: pgmap v141: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:43 smithi162 ceph-mon[130941]: pgmap v142: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:43 smithi183 ceph-mon[131414]: pgmap v142: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:43 smithi145 ceph-mon[97681]: pgmap v142: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:45 smithi162 ceph-mon[130941]: pgmap v143: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:45 smithi183 ceph-mon[131414]: pgmap v143: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:46.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:45 smithi145 ceph-mon[97681]: pgmap v143: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:47 smithi162 ceph-mon[130941]: pgmap v144: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:47 smithi183 ceph-mon[131414]: pgmap v144: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:48.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:47 smithi145 ceph-mon[97681]: pgmap v144: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:49.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:48 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:49:49.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:48 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:49:49.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:48 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:49:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:49 smithi162 ceph-mon[130941]: pgmap v145: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:49 smithi183 ceph-mon[131414]: pgmap v145: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:50.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:49 smithi145 ceph-mon[97681]: pgmap v145: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:51 smithi183 ceph-mon[131414]: pgmap v146: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:52.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:51 smithi145 ceph-mon[97681]: pgmap v146: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:52.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:51 smithi162 ceph-mon[130941]: pgmap v146: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:53 smithi183 ceph-mon[131414]: pgmap v147: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:54.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:53 smithi145 ceph-mon[97681]: pgmap v147: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:53 smithi162 ceph-mon[130941]: pgmap v147: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:55 smithi183 ceph-mon[131414]: pgmap v148: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:56.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:55 smithi145 ceph-mon[97681]: pgmap v148: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:56.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:55 smithi162 ceph-mon[130941]: pgmap v148: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:49:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:57 smithi183 ceph-mon[131414]: pgmap v149: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:57 smithi145 ceph-mon[97681]: pgmap v149: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:57 smithi162 ceph-mon[130941]: pgmap v149: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:49:59.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:58 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:49:59.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:58 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:49:59.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:58 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:00.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:49:59 smithi145 ceph-mon[97681]: pgmap v150: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:49:59 smithi162 ceph-mon[130941]: pgmap v150: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:49:59 smithi183 ceph-mon[131414]: pgmap v150: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:01.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:00 smithi145 ceph-mon[97681]: overall HEALTH_OK 2023-08-25T04:50:01.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:00 smithi162 ceph-mon[130941]: overall HEALTH_OK 2023-08-25T04:50:01.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:00 smithi183 ceph-mon[131414]: overall HEALTH_OK 2023-08-25T04:50:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:01 smithi145 ceph-mon[97681]: pgmap v151: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:02.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:01 smithi162 ceph-mon[130941]: pgmap v151: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:01 smithi183 ceph-mon[131414]: pgmap v151: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:03 smithi145 ceph-mon[97681]: pgmap v152: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:04.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:03 smithi162 ceph-mon[130941]: pgmap v152: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:03 smithi183 ceph-mon[131414]: pgmap v152: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:05.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:50:05.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 -- 172.21.15.145:0/437893477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc1068a0 msgr2=0x7fdccc106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/437893477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc1068a0 0x7fdccc106c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009990 tx=0x7fdcbc009ca0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 -- 172.21.15.145:0/437893477 shutdown_connections 2023-08-25T04:50:05.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/437893477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc101250 0x7fdccc10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.470 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/437893477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc1068a0 0x7fdccc106c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.470 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/437893477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc100890 0x7fdccc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.471 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 -- 172.21.15.145:0/437893477 >> 172.21.15.145:0/437893477 conn(0x7fdccc0fc380 msgr2=0x7fdccc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.471 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 -- 172.21.15.145:0/437893477 shutdown_connections 2023-08-25T04:50:05.471 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.467+0000 7fdcd0fff700 1 -- 172.21.15.145:0/437893477 wait complete. 2023-08-25T04:50:05.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 Processor -- start 2023-08-25T04:50:05.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 -- start start 2023-08-25T04:50:05.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.476 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 0x7fdccc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.477 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.477 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc199040 con 0x7fdccc1068a0 2023-08-25T04:50:05.477 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1991b0 con 0x7fdccc100890 2023-08-25T04:50:05.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.469+0000 7fdcd0fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc199370 con 0x7fdccc101250 2023-08-25T04:50:05.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 0x7fdccc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:60392/0 (socket says 172.21.15.145:60392) 2023-08-25T04:50:05.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcca59c700 1 -- 172.21.15.145:0/3192129476 learned_addr learned my addr 172.21.15.145:0/3192129476 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:05.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38842/0 (socket says 172.21.15.145:38842) 2023-08-25T04:50:05.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 0x7fdccc19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50372/0 (socket says 172.21.15.145:50372) 2023-08-25T04:50:05.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 -- 172.21.15.145:0/3192129476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 msgr2=0x7fdccc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 0x7fdccc19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 -- 172.21.15.145:0/3192129476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 msgr2=0x7fdccc19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 -- 172.21.15.145:0/3192129476 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0095f0 con 0x7fdccc1068a0 2023-08-25T04:50:05.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdcca59c700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:50:05.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.470+0000 7fdccad9d700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007f10 tx=0x7fdcc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.471+0000 7fdcbb7fe700 1 -- 172.21.15.145:0/3192129476 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4009ea0 con 0x7fdccc1068a0 2023-08-25T04:50:05.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.471+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc199650 con 0x7fdccc1068a0 2023-08-25T04:50:05.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.471+0000 7fdcbb7fe700 1 -- 172.21.15.145:0/3192129476 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc400f040 con 0x7fdccc1068a0 2023-08-25T04:50:05.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.471+0000 7fdcbb7fe700 1 -- 172.21.15.145:0/3192129476 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4004350 con 0x7fdccc1068a0 2023-08-25T04:50:05.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.471+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc1a5950 con 0x7fdccc1068a0 2023-08-25T04:50:05.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.475+0000 7fdcbb7fe700 1 -- 172.21.15.145:0/3192129476 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fdcc400db20 con 0x7fdccc1068a0 2023-08-25T04:50:05.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.475+0000 7fdce48c5b80 1 Processor -- start 2023-08-25T04:50:05.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 -- start start 2023-08-25T04:50:05.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788535700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d788535c40 0x55d7885360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78849d7b0 con 0x55d78853bca0 2023-08-25T04:50:05.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78848e780 con 0x55d788535c40 2023-08-25T04:50:05.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78848c0a0 con 0x55d78853b380 2023-08-25T04:50:05.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788535700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788535700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38856/0 (socket says 172.21.15.145:38856) 2023-08-25T04:50:05.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdcca59c700 1 -- 172.21.15.145:0/3517223482 learned_addr learned my addr 172.21.15.145:0/3517223482 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:05.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.476+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d788535c40 0x55d7885360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/3517223482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d788535c40 msgr2=0x55d7885360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d788535c40 0x55d7885360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/3517223482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 msgr2=0x55d788535700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788535700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/3517223482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d788307630 con 0x55d78853b380 2023-08-25T04:50:05.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853b760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdccc10db90 tx=0x7fdcc0009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeaf7c9b4b8057d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcbb7fe700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 0x7fdcb40a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcb8ff9700 1 -- 172.21.15.145:0/3517223482 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0004050 con 0x55d78853b380 2023-08-25T04:50:05.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcb8ff9700 1 -- 172.21.15.145:0/3517223482 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc0011e90 con 0x55d78853b380 2023-08-25T04:50:05.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.477+0000 7fdcb8ff9700 1 -- 172.21.15.145:0/3517223482 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0005280 con 0x55d78853b380 2023-08-25T04:50:05.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdcbb7fe700 1 -- 172.21.15.145:0/3192129476 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fdcc40cb0d0 con 0x7fdccc1068a0 2023-08-25T04:50:05.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdcca59c700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 0x7fdcb40a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 -- 172.21.15.145:0/3517223482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 msgr2=0x55d78853b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853b760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdccc10db90 tx=0x7fdcc0009750 comp rx=0 tx=0).stop 2023-08-25T04:50:05.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 -- 172.21.15.145:0/3517223482 shutdown_connections 2023-08-25T04:50:05.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d788535c40 0x55d7885360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853b760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/3517223482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788535700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.478+0000 7fdce48c5b80 1 -- 172.21.15.145:0/3517223482 >> 172.21.15.145:0/3517223482 conn(0x55d7884aa0e0 msgr2=0x55d788530410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.479+0000 7fdcca59c700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 0x7fdcb40a2600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55d78853c100 tx=0x7fdcbc00b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.479+0000 7fdce48c5b80 1 -- 172.21.15.145:0/3517223482 shutdown_connections 2023-08-25T04:50:05.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.479+0000 7fdce48c5b80 1 -- 172.21.15.145:0/3517223482 wait complete. 2023-08-25T04:50:05.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.479+0000 7fdce48c5b80 1 -- start start 2023-08-25T04:50:05.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.479+0000 7fdce48c5b80 1 Processor -- start 2023-08-25T04:50:05.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788541c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d78853f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788487990 con 0x55d788535c40 2023-08-25T04:50:05.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788425bb0 con 0x55d78853bca0 2023-08-25T04:50:05.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7884258b0 con 0x55d78853b380 2023-08-25T04:50:05.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdce48c5b80 10 client.0 _ll_register_callbacks cb 0x55d78836fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:50:05.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50378/0 (socket says 172.21.15.145:50378) 2023-08-25T04:50:05.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/127365963 learned_addr learned my addr 172.21.15.145:0/127365963 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:05.501 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdcca59c700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788541c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.501 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.480+0000 7fdccad9d700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d78853f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.502 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/127365963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 msgr2=0x55d78853f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.502 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d78853f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.502 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 msgr2=0x55d788541c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788541c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/127365963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc0019040 con 0x55d78853b380 2023-08-25T04:50:05.503 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdccad9d700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d78853f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:50:05.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004ce0 tx=0x7fdcc0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c33d0e7eb16afc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.504 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:50:05.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0024070 con 0x55d78853b380 2023-08-25T04:50:05.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7885051c0 con 0x55d78853b380 2023-08-25T04:50:05.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.481+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fdcc0004030 con 0x55d78853b380 2023-08-25T04:50:05.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc0004b00 con 0x55d78853b380 2023-08-25T04:50:05.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 10 client.25924 fetch_fsmap learned FSMap version 31 2023-08-25T04:50:05.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d7882b3de0 con 0x55d78853b380 2023-08-25T04:50:05.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7885051c0 con 0x55d78853b380 2023-08-25T04:50:05.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdcc001d070 con 0x55d78853b380 2023-08-25T04:50:05.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 10 client.25924 fetch_fsmap finished waiting for FSMap version 31 2023-08-25T04:50:05.508 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 10 client.25924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:50:05.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 20 client.25924 populate_metadata read hostname 'smithi145' 2023-08-25T04:50:05.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d78854f6d0 0x55d788551af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.509 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 4 client.25924 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:50:05.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.482+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d78842dba0 con 0x55d78854f6d0 2023-08-25T04:50:05.510 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.483+0000 7fdcca59c700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d78854f6d0 0x55d788551af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.483+0000 7fdcb1ffb700 10 client.25924 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:05.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.483+0000 7fdcca59c700 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d78854f6d0 0x55d788551af0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d2e7d8599b8bab server_cookie=4df164210e052b16 in_seq=0 out_seq=0 2023-08-25T04:50:05.511 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/127365963 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d78842dba0 con 0x55d78854f6d0 2023-08-25T04:50:05.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdcb1ffb700 10 client.25924 handle_command_reply: tid=0 2023-08-25T04:50:05.512 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 client.25924 shutdown 2023-08-25T04:50:05.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 msgr2=0x55d78853ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004ce0 tx=0x7fdcc0009260 comp rx=0 tx=0).stop 2023-08-25T04:50:05.513 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 shutdown_connections 2023-08-25T04:50:05.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d78853f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.514 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788541c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d78854f6d0 0x55d788551af0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/127365963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d78853ee80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdcb1ffb700 0 client.25924 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:05.515 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 >> 172.21.15.145:0/127365963 conn(0x55d7884aa0e0 msgr2=0x55d78852ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 shutdown_connections 2023-08-25T04:50:05.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.484+0000 7fdce48c5b80 1 -- 172.21.15.145:0/127365963 wait complete. 2023-08-25T04:50:05.516 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.485+0000 7fdce48c5b80 20 client.25924 trim_cache size 0 max 16384 2023-08-25T04:50:05.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 Processor -- start 2023-08-25T04:50:05.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 -- start start 2023-08-25T04:50:05.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788535700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.522 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d788541ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788309920 con 0x55d788535c40 2023-08-25T04:50:05.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78852c810 con 0x55d78853bca0 2023-08-25T04:50:05.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788552150 con 0x55d78853b380 2023-08-25T04:50:05.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788535700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdcc9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50384/0 (socket says 172.21.15.145:50384) 2023-08-25T04:50:05.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/1113427106 learned_addr learned my addr 172.21.15.145:0/1113427106 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:05.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.506+0000 7fdccad9d700 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d788541ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/1113427106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 msgr2=0x55d788541ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d788541ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/1113427106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 msgr2=0x55d788535700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.527 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788535700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 -- 172.21.15.145:0/1113427106 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7885566e0 con 0x55d78853b380 2023-08-25T04:50:05.528 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcca59c700 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788535700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:50:05.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0001e90 tx=0x7fdcc00092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ddc835898f4f36c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.529 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcb97fa700 1 -- 172.21.15.145:0/1113427106 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0024040 con 0x55d78853b380 2023-08-25T04:50:05.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcb97fa700 1 -- 172.21.15.145:0/1113427106 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc0011e90 con 0x55d78853b380 2023-08-25T04:50:05.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.507+0000 7fdcb97fa700 1 -- 172.21.15.145:0/1113427106 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000fe70 con 0x55d78853b380 2023-08-25T04:50:05.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1113427106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 msgr2=0x55d7885417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0001e90 tx=0x7fdcc00092d0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.530 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1113427106 shutdown_connections 2023-08-25T04:50:05.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853bca0 0x55d788541ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d788535c40 0x55d788535700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1113427106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78853b380 0x55d7885417a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.531 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1113427106 >> 172.21.15.145:0/1113427106 conn(0x55d7884aa0e0 msgr2=0x55d788538fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1113427106 shutdown_connections 2023-08-25T04:50:05.532 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.508+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1113427106 wait complete. 2023-08-25T04:50:05.533 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.509+0000 7fdce48c5b80 1 -- start start 2023-08-25T04:50:05.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.509+0000 7fdce48c5b80 1 Processor -- start 2023-08-25T04:50:05.534 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853b380 0x55d788536e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.535 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788537430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788537ad0 con 0x55d78853bca0 2023-08-25T04:50:05.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788537c40 con 0x55d78853b380 2023-08-25T04:50:05.536 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788537e00 con 0x55d788535c40 2023-08-25T04:50:05.537 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdce48c5b80 10 client.0 _ll_register_callbacks cb 0x7fdcc0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:50:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50390/0 (socket says 172.21.15.145:50390) 2023-08-25T04:50:05.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 -- 172.21.15.145:0/1414338692 learned_addr learned my addr 172.21.15.145:0/1414338692 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:05.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853b380 0x55d788536e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdccad9d700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788537430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 -- 172.21.15.145:0/1414338692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853b380 msgr2=0x55d788536e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853b380 0x55d788536e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 -- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 msgr2=0x55d788537430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:05.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788537430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.510+0000 7fdcca59c700 1 -- 172.21.15.145:0/1414338692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc10e060 con 0x55d788535c40 2023-08-25T04:50:05.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdcca59c700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005910 tx=0x7fdcbc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fdf8f699d0314fd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:05.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdcb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:50:05.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0374c0 con 0x55d788535c40 2023-08-25T04:50:05.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7fdcbc0048e0 con 0x55d788535c40 2023-08-25T04:50:05.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7885288b0 con 0x55d788535c40 2023-08-25T04:50:05.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcbc004aa0 con 0x55d788535c40 2023-08-25T04:50:05.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdce48c5b80 10 client.25930 fetch_fsmap learned FSMap version 31 2023-08-25T04:50:05.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.511+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d7882b3de0 con 0x55d788535c40 2023-08-25T04:50:05.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7885288b0 con 0x55d788535c40 2023-08-25T04:50:05.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdcbc00fb80 con 0x55d788535c40 2023-08-25T04:50:05.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 31 2023-08-25T04:50:05.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:50:05.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 20 client.25930 populate_metadata read hostname 'smithi145' 2023-08-25T04:50:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7885a3cb0 0x55d7885a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 4 client.25930 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:50:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d788480970 con 0x55d7885a3cb0 2023-08-25T04:50:05.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.512+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7885a3cb0 0x55d7885a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:05.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdcc9d9b700 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7885a3cb0 0x55d7885a60d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcd2a8fae17667e server_cookie=30cf49222e583f31 in_seq=0 out_seq=0 2023-08-25T04:50:05.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdcb1ffb700 10 client.25930 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:05.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdcb1ffb700 1 -- 172.21.15.145:0/1414338692 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d788480970 con 0x55d7885a3cb0 2023-08-25T04:50:05.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdcb1ffb700 10 client.25930 handle_command_reply: tid=0 2023-08-25T04:50:05.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 client.25930 shutdown 2023-08-25T04:50:05.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 msgr2=0x55d788546a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005910 tx=0x7fdcbc009310 comp rx=0 tx=0).stop 2023-08-25T04:50:05.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 shutdown_connections 2023-08-25T04:50:05.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d78853bca0 0x55d788537430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7885a3cb0 0x55d7885a60d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d788535c40 0x55d788546a20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 --2- 172.21.15.145:0/1414338692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d78853b380 0x55d788536e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 >> 172.21.15.145:0/1414338692 conn(0x55d7884aa0e0 msgr2=0x55d78853e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdcb1ffb700 0 client.25930 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:05.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 shutdown_connections 2023-08-25T04:50:05.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.513+0000 7fdce48c5b80 1 -- 172.21.15.145:0/1414338692 wait complete. 2023-08-25T04:50:05.550 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdce48c5b80 20 client.25930 trim_cache size 0 max 16384 2023-08-25T04:50:05.551 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:50:05.551 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:50:05.552 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:50:05.552 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:50:05.552 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:50:05.552 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:50:05.552 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:50:05.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:50:05.553 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:50:05.553 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:50:05.554 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 msgr2=0x7fdcb40a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 0x7fdcb40a2600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55d78853c100 tx=0x7fdcbc00b260 comp rx=0 tx=0).stop 2023-08-25T04:50:05.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 msgr2=0x7fdccc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:05.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007f10 tx=0x7fdcc400d3b0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.555 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 shutdown_connections 2023-08-25T04:50:05.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdccc1068a0 0x7fdccc1989d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdcb40a0140 0x7fdcb40a2600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdccc100890 0x7fdccc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.556 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 --2- 172.21.15.145:0/3192129476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdccc101250 0x7fdccc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:05.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 >> 172.21.15.145:0/3192129476 conn(0x7fdccc0fc380 msgr2=0x7fdccc10bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:05.557 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 shutdown_connections 2023-08-25T04:50:05.558 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:05.514+0000 7fdcd0fff700 1 -- 172.21.15.145:0/3192129476 wait complete. 2023-08-25T04:50:05.558 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:50:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:05 smithi145 ceph-mon[97681]: pgmap v153: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:06.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:05 smithi162 ceph-mon[130941]: pgmap v153: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:05 smithi183 ceph-mon[131414]: pgmap v153: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:07 smithi145 ceph-mon[97681]: pgmap v154: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:08.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:07 smithi162 ceph-mon[130941]: pgmap v154: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:07 smithi183 ceph-mon[131414]: pgmap v154: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:50:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:09 smithi145 ceph-mon[97681]: pgmap v155: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:10.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:09 smithi162 ceph-mon[130941]: pgmap v155: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:10.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:09 smithi183 ceph-mon[131414]: pgmap v155: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:50:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:11 smithi145 ceph-mon[97681]: pgmap v156: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:50:12.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:11 smithi162 ceph-mon[130941]: pgmap v156: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:50:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:11 smithi183 ceph-mon[131414]: pgmap v156: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:50:14.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:13 smithi145 ceph-mon[97681]: pgmap v157: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:14.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:13 smithi162 ceph-mon[130941]: pgmap v157: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:13 smithi183 ceph-mon[131414]: pgmap v157: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:15 smithi145 ceph-mon[97681]: pgmap v158: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:16.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:15 smithi162 ceph-mon[130941]: pgmap v158: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:16.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:15 smithi183 ceph-mon[131414]: pgmap v158: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:50:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:17 smithi145 ceph-mon[97681]: pgmap v159: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:50:18.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:17 smithi162 ceph-mon[130941]: pgmap v159: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:50:18.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:17 smithi183 ceph-mon[131414]: pgmap v159: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:50:19.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:18 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:50:19.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:18 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:50:19.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:18 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:50:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:19 smithi145 ceph-mon[97681]: pgmap v160: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:50:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:19 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:50:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:19 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:50:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:19 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:50:20.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:19 smithi162 ceph-mon[130941]: pgmap v160: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:50:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:19 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:50:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:19 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:50:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:19 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:50:20.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:19 smithi183 ceph-mon[131414]: pgmap v160: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:50:20.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:19 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:50:20.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:19 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:50:20.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:19 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:50:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:21 smithi145 ceph-mon[97681]: pgmap v161: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:50:22.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:21 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:22.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:21 smithi162 ceph-mon[130941]: pgmap v161: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:50:22.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:21 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:21 smithi183 ceph-mon[131414]: pgmap v161: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:50:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:21 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:23 smithi145 ceph-mon[97681]: pgmap v162: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:24.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:23 smithi162 ceph-mon[130941]: pgmap v162: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:24.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:23 smithi183 ceph-mon[131414]: pgmap v162: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:26.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:25 smithi145 ceph-mon[97681]: pgmap v163: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:26.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:25 smithi162 ceph-mon[130941]: pgmap v163: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:26.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:25 smithi183 ceph-mon[131414]: pgmap v163: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:27 smithi145 ceph-mon[97681]: pgmap v164: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:28.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:27 smithi162 ceph-mon[130941]: pgmap v164: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:28.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:27 smithi183 ceph-mon[131414]: pgmap v164: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:30.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:29 smithi145 ceph-mon[97681]: pgmap v165: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:30.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:29 smithi162 ceph-mon[130941]: pgmap v165: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:30.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:29 smithi183 ceph-mon[131414]: pgmap v165: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:31 smithi145 ceph-mon[97681]: pgmap v166: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:31 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:32.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:31 smithi162 ceph-mon[130941]: pgmap v166: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:31 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:32.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:31 smithi183 ceph-mon[131414]: pgmap v166: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:32.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:31 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:34 smithi145 ceph-mon[97681]: pgmap v167: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:34.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:34 smithi162 ceph-mon[130941]: pgmap v167: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:34.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:34 smithi183 ceph-mon[131414]: pgmap v167: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:35.559 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:50:35.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 -- 172.21.15.145:0/2979420158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 msgr2=0x7f1b4010d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/2979420158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b4010d5c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b0a0 tx=0x7f1b3800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 -- 172.21.15.145:0/2979420158 shutdown_connections 2023-08-25T04:50:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/2979420158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b4010d5c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/2979420158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b40106700 0x7f1b40106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/2979420158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b401006f0 0x7f1b40100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 -- 172.21.15.145:0/2979420158 >> 172.21.15.145:0/2979420158 conn(0x7f1b400fc1c0 msgr2=0x7f1b400fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 -- 172.21.15.145:0/2979420158 shutdown_connections 2023-08-25T04:50:35.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.791+0000 7f1b45d9d700 1 -- 172.21.15.145:0/2979420158 wait complete. 2023-08-25T04:50:35.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 Processor -- start 2023-08-25T04:50:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 -- start start 2023-08-25T04:50:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 0x7f1b40196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b40106700 0x7f1b4019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019b650 con 0x7f1b40106700 2023-08-25T04:50:35.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019b7c0 con 0x7f1b401006f0 2023-08-25T04:50:35.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b45d9d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019b980 con 0x7f1b401010b0 2023-08-25T04:50:35.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.793+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:58534/0 (socket says 172.21.15.145:58534) 2023-08-25T04:50:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 -- 172.21.15.145:0/275539892 learned_addr learned my addr 172.21.15.145:0/275539892 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b4559c700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b40106700 0x7f1b4019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 0x7f1b40196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 -- 172.21.15.145:0/275539892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 msgr2=0x7f1b40196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 0x7f1b40196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 -- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b40106700 msgr2=0x7f1b4019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b40106700 0x7f1b4019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 -- 172.21.15.145:0/275539892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38009d00 con 0x7f1b401010b0 2023-08-25T04:50:35.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 0x7f1b40196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:50:35.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9a0 tx=0x7f1b2c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3dffb700 1 -- 172.21.15.145:0/275539892 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c009790 con 0x7f1b401010b0 2023-08-25T04:50:35.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3dffb700 1 -- 172.21.15.145:0/275539892 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2c00cbd0 con 0x7f1b401010b0 2023-08-25T04:50:35.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b3dffb700 1 -- 172.21.15.145:0/275539892 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c018680 con 0x7f1b401010b0 2023-08-25T04:50:35.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.794+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4019bc60 con 0x7f1b401010b0 2023-08-25T04:50:35.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.795+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b401a34a0 con 0x7f1b401010b0 2023-08-25T04:50:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.799+0000 7f1b3dffb700 1 -- 172.21.15.145:0/275539892 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f1b2c00cd40 con 0x7f1b401010b0 2023-08-25T04:50:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.800+0000 7f1b5a83eb80 1 Processor -- start 2023-08-25T04:50:35.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.800+0000 7f1b5a83eb80 1 -- start start 2023-08-25T04:50:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 0x55573ef3f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573eea67b0 con 0x55573ef3ec40 2023-08-25T04:50:35.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ee97780 con 0x55573ef44e20 2023-08-25T04:50:35.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ee950a0 con 0x55573ef44500 2023-08-25T04:50:35.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:58548/0 (socket says 172.21.15.145:58548) 2023-08-25T04:50:35.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 -- 172.21.15.145:0/3255342605 learned_addr learned my addr 172.21.15.145:0/3255342605 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:35.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 0x55573ef3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 -- 172.21.15.145:0/3255342605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 msgr2=0x55573ef3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 -- 172.21.15.145:0/3255342605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 msgr2=0x55573ef3f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 0x55573ef3f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.801+0000 7f1b4559c700 1 -- 172.21.15.145:0/3255342605 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55573ed10630 con 0x55573ef44500 2023-08-25T04:50:35.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 0x55573ef3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:50:35.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b3dffb700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 0x7f1b300a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b38015000 tx=0x7f1b38009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4501f61cf38348c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b3dffb700 1 -- 172.21.15.145:0/275539892 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1b2c014070 con 0x7f1b401010b0 2023-08-25T04:50:35.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b2affd700 1 -- 172.21.15.145:0/3255342605 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800e040 con 0x55573ef44500 2023-08-25T04:50:35.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b2affd700 1 -- 172.21.15.145:0/3255342605 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3800bdb0 con 0x55573ef44500 2023-08-25T04:50:35.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 0x7f1b300a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.812 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.802+0000 7f1b2affd700 1 -- 172.21.15.145:0/3255342605 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3801c6a0 con 0x55573ef44500 2023-08-25T04:50:35.819 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3255342605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 msgr2=0x55573ef448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 0x7f1b300a26a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55573ef452a0 tx=0x7f1b34009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.820 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b38015000 tx=0x7f1b38009450 comp rx=0 tx=0).stop 2023-08-25T04:50:35.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3255342605 shutdown_connections 2023-08-25T04:50:35.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef3ec40 0x55573ef3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.821 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef448e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3255342605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3255342605 >> 172.21.15.145:0/3255342605 conn(0x55573eeb30e0 msgr2=0x55573ef39940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.822 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3255342605 shutdown_connections 2023-08-25T04:50:35.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.803+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3255342605 wait complete. 2023-08-25T04:50:35.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.804+0000 7f1b5a83eb80 1 -- start start 2023-08-25T04:50:35.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.804+0000 7f1b5a83eb80 1 Processor -- start 2023-08-25T04:50:35.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.823 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.824 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 0x55573ef4cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.824 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ee90990 con 0x55573ef44e20 2023-08-25T04:50:35.824 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ee2ebb0 con 0x55573ef3ec40 2023-08-25T04:50:35.824 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ee2e8b0 con 0x55573ef44500 2023-08-25T04:50:35.825 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b5a83eb80 10 client.0 _ll_register_callbacks cb 0x55573ed78a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:50:35.825 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.825 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:58560/0 (socket says 172.21.15.145:58560) 2023-08-25T04:50:35.826 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b3ffff700 1 -- 172.21.15.145:0/924384197 learned_addr learned my addr 172.21.15.145:0/924384197 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:35.826 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.805+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.827 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b4559c700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 0x55573ef4cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.827 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 -- 172.21.15.145:0/924384197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 msgr2=0x55573ef52870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.827 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.828 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 -- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 msgr2=0x55573ef4cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.828 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 0x55573ef4cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.829 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 -- 172.21.15.145:0/924384197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4010dec0 con 0x55573ef44500 2023-08-25T04:50:35.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:50:35.830 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b4559c700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 0x55573ef4cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:50:35.831 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b3ffff700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e970 tx=0x7f1b2c0c9530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dd289871b72315 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.831 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:50:35.831 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c09cdc0 con 0x55573ef44500 2023-08-25T04:50:35.832 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55573ef0e1c0 con 0x55573ef44500 2023-08-25T04:50:35.832 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1b2c0a0da0 con 0x55573ef44500 2023-08-25T04:50:35.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b2c0b1150 con 0x55573ef44500 2023-08-25T04:50:35.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b5a83eb80 10 client.25942 fetch_fsmap learned FSMap version 33 2023-08-25T04:50:35.833 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.806+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55573ecbcde0 con 0x55573ef44500 2023-08-25T04:50:35.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55573ef0e1c0 con 0x55573ef44500 2023-08-25T04:50:35.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1b2c097ef0 con 0x55573ef44500 2023-08-25T04:50:35.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 10 client.25942 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T04:50:35.834 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:50:35.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 20 client.25942 populate_metadata read hostname 'smithi145' 2023-08-25T04:50:35.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573ef5cbd0 0x55573ef5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 4 client.25942 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:50:35.835 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55573ee36ba0 con 0x55573ef5cbd0 2023-08-25T04:50:35.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.807+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573ef5cbd0 0x55573ef5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.808+0000 7f1b1ffff700 10 client.25942 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:35.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.808+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573ef5cbd0 0x55573ef5eff0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1e32b692b0cd12 server_cookie=31bf03c820272623 in_seq=0 out_seq=0 2023-08-25T04:50:35.836 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.808+0000 7f1b1ffff700 1 -- 172.21.15.145:0/924384197 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55573ee36ba0 con 0x55573ef5cbd0 2023-08-25T04:50:35.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.808+0000 7f1b1ffff700 10 client.25942 handle_command_reply: tid=0 2023-08-25T04:50:35.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.808+0000 7f1b5a83eb80 1 client.25942 shutdown 2023-08-25T04:50:35.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 msgr2=0x55573ef52db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e970 tx=0x7f1b2c0c9530 comp rx=0 tx=0).stop 2023-08-25T04:50:35.837 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 shutdown_connections 2023-08-25T04:50:35.838 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44e20 0x55573ef4cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.838 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.838 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573ef5cbd0 0x55573ef5eff0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.838 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/924384197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44500 0x55573ef52db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.838 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b1ffff700 0 client.25942 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:35.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 >> 172.21.15.145:0/924384197 conn(0x55573eeb30e0 msgr2=0x55573ef3d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 shutdown_connections 2023-08-25T04:50:35.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/924384197 wait complete. 2023-08-25T04:50:35.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.809+0000 7f1b5a83eb80 20 client.25942 trim_cache size 0 max 16384 2023-08-25T04:50:35.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.833+0000 7f1b5a83eb80 1 Processor -- start 2023-08-25T04:50:35.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.833+0000 7f1b5a83eb80 1 -- start start 2023-08-25T04:50:35.847 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.833+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.833+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef4b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.848 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.833+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef4be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ed12920 con 0x55573ef44500 2023-08-25T04:50:35.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ef3f610 con 0x55573ef44e20 2023-08-25T04:50:35.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ef3a530 con 0x55573ef3ec40 2023-08-25T04:50:35.849 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:58576/0 (socket says 172.21.15.145:58576) 2023-08-25T04:50:35.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef4b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 -- 172.21.15.145:0/1062943043 learned_addr learned my addr 172.21.15.145:0/1062943043 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:35.850 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b4559c700 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef4be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 -- 172.21.15.145:0/1062943043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 msgr2=0x55573ef4be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef4be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 -- 172.21.15.145:0/1062943043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 msgr2=0x55573ef4b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.851 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef4b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 -- 172.21.15.145:0/1062943043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55573eca5c60 con 0x55573ef3ec40 2023-08-25T04:50:35.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b4559c700 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef4be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:50:35.852 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.834+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009810 tx=0x7f1b34015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83816e843ef9d097 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b2b7fe700 1 -- 172.21.15.145:0/1062943043 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34022040 con 0x55573ef3ec40 2023-08-25T04:50:35.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b2b7fe700 1 -- 172.21.15.145:0/1062943043 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b34010460 con 0x55573ef3ec40 2023-08-25T04:50:35.853 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b2b7fe700 1 -- 172.21.15.145:0/1062943043 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b340038e0 con 0x55573ef3ec40 2023-08-25T04:50:35.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/1062943043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 msgr2=0x55573ef41040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009810 tx=0x7f1b34015000 comp rx=0 tx=0).stop 2023-08-25T04:50:35.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/1062943043 shutdown_connections 2023-08-25T04:50:35.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef44e20 0x55573ef4be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.854 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef3ec40 0x55573ef41040 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.855 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/1062943043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef4b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.855 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.835+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/1062943043 >> 172.21.15.145:0/1062943043 conn(0x55573eeb30e0 msgr2=0x55573ef438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.855 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.836+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/1062943043 shutdown_connections 2023-08-25T04:50:35.855 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.836+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/1062943043 wait complete. 2023-08-25T04:50:35.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.836+0000 7f1b5a83eb80 1 -- start start 2023-08-25T04:50:35.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.836+0000 7f1b5a83eb80 1 Processor -- start 2023-08-25T04:50:35.856 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef50c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.857 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ef51700 con 0x55573ef44500 2023-08-25T04:50:35.858 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ef51870 con 0x55573ef3ec40 2023-08-25T04:50:35.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573ef51a30 con 0x55573ef44e20 2023-08-25T04:50:35.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b5a83eb80 10 client.0 _ll_register_callbacks cb 0x7f1b2c098d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:50:35.859 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef50c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.860 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef50c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40460/0 (socket says 172.21.15.145:40460) 2023-08-25T04:50:35.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:58578/0 (socket says 172.21.15.145:58578) 2023-08-25T04:50:35.861 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b3ffff700 1 -- 172.21.15.145:0/3900311211 learned_addr learned my addr 172.21.15.145:0/3900311211 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:50:35.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.862 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 -- 172.21.15.145:0/3900311211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 msgr2=0x55573ef52a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.863 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 -- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 msgr2=0x55573ef50c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:50:35.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef50c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.864 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.837+0000 7f1b4559c700 1 -- 172.21.15.145:0/3900311211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b34027900 con 0x55573ef44e20 2023-08-25T04:50:35.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b4559c700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55573ef35d60 tx=0x7f1b38003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf4ce13043aad37 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:50:35.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:50:35.865 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800e070 con 0x55573ef44e20 2023-08-25T04:50:35.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55573ee89970 con 0x55573ef44e20 2023-08-25T04:50:35.866 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f1b380076a0 con 0x55573ef44e20 2023-08-25T04:50:35.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b3800b780 con 0x55573ef44e20 2023-08-25T04:50:35.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b5a83eb80 10 client.25948 fetch_fsmap learned FSMap version 33 2023-08-25T04:50:35.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55573ece3250 con 0x55573ef44e20 2023-08-25T04:50:35.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.838+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55573ee89970 con 0x55573ef44e20 2023-08-25T04:50:35.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1b38019030 con 0x55573ef44e20 2023-08-25T04:50:35.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 10 client.25948 fetch_fsmap finished waiting for FSMap version 33 2023-08-25T04:50:35.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:50:35.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 20 client.25948 populate_metadata read hostname 'smithi145' 2023-08-25T04:50:35.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573efb1a70 0x55573efb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:50:35.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 4 client.25948 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:50:35.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55573ee4a050 con 0x55573efb1a70 2023-08-25T04:50:35.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573efb1a70 0x55573efb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:50:35.869 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b1ffff700 10 client.25948 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:35.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.839+0000 7f1b44d9b700 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573efb1a70 0x55573efb3e60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ca34045b1368e8 server_cookie=bbbae676e7f85127 in_seq=0 out_seq=0 2023-08-25T04:50:35.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b1ffff700 1 -- 172.21.15.145:0/3900311211 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55573ee4a050 con 0x55573efb1a70 2023-08-25T04:50:35.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b1ffff700 10 client.25948 handle_command_reply: tid=0 2023-08-25T04:50:35.870 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 client.25948 shutdown 2023-08-25T04:50:35.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 msgr2=0x55573ef511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55573ef35d60 tx=0x7f1b38003c30 comp rx=0 tx=0).stop 2023-08-25T04:50:35.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 shutdown_connections 2023-08-25T04:50:35.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55573efb1a70 0x55573efb3e60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55573ef44e20 0x55573ef511c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55573ef3ec40 0x55573ef52a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b1ffff700 0 client.25948 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:50:35.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 --2- 172.21.15.145:0/3900311211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55573ef44500 0x55573ef50c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 >> 172.21.15.145:0/3900311211 conn(0x55573eeb30e0 msgr2=0x55573ef390a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 shutdown_connections 2023-08-25T04:50:35.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 1 -- 172.21.15.145:0/3900311211 wait complete. 2023-08-25T04:50:35.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.840+0000 7f1b5a83eb80 20 client.25948 trim_cache size 0 max 16384 2023-08-25T04:50:35.873 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:50:35.874 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:50:35.874 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:50:35.874 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:50:35.874 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:50:35.875 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:50:35.875 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:50:35.875 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:50:35.875 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:50:35.875 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:50:35.876 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 msgr2=0x7f1b300a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 0x7f1b300a26a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55573ef452a0 tx=0x7f1b34009260 comp rx=0 tx=0).stop 2023-08-25T04:50:35.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 msgr2=0x7f1b40196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:50:35.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9a0 tx=0x7f1b2c00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.877 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 shutdown_connections 2023-08-25T04:50:35.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b40106700 0x7f1b4019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f1b300a01e0 0x7f1b300a26a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b401006f0 0x7f1b40196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 --2- 172.21.15.145:0/275539892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b401010b0 0x7f1b40196bb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:50:35.878 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 >> 172.21.15.145:0/275539892 conn(0x7f1b400fc1c0 msgr2=0x7f1b400fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:50:35.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 shutdown_connections 2023-08-25T04:50:35.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:50:35.841+0000 7f1b45d9d700 1 -- 172.21.15.145:0/275539892 wait complete. 2023-08-25T04:50:35.879 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:50:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:36 smithi145 ceph-mon[97681]: pgmap v168: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:36 smithi162 ceph-mon[130941]: pgmap v168: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:36 smithi183 ceph-mon[131414]: pgmap v168: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:38.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:38 smithi145 ceph-mon[97681]: pgmap v169: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:38 smithi162 ceph-mon[130941]: pgmap v169: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:38 smithi183 ceph-mon[131414]: pgmap v169: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:40 smithi145 ceph-mon[97681]: pgmap v170: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:40.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:40 smithi162 ceph-mon[130941]: pgmap v170: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:40.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:40 smithi183 ceph-mon[131414]: pgmap v170: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:42.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:42 smithi145 ceph-mon[97681]: pgmap v171: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:42 smithi162 ceph-mon[130941]: pgmap v171: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:42.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:42 smithi183 ceph-mon[131414]: pgmap v171: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:44 smithi145 ceph-mon[97681]: pgmap v172: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:44.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:44 smithi162 ceph-mon[130941]: pgmap v172: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:44.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:44 smithi183 ceph-mon[131414]: pgmap v172: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:46.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:45 smithi145 ceph-mon[97681]: pgmap v173: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:46.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:45 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:45 smithi162 ceph-mon[130941]: pgmap v173: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:46.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:45 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:45 smithi183 ceph-mon[131414]: pgmap v173: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:46.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:45 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:50:48.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:47 smithi145 ceph-mon[97681]: pgmap v174: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:48.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:47 smithi162 ceph-mon[130941]: pgmap v174: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:47 smithi183 ceph-mon[131414]: pgmap v174: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:49 smithi145 ceph-mon[97681]: pgmap v175: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:49 smithi162 ceph-mon[130941]: pgmap v175: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:49 smithi183 ceph-mon[131414]: pgmap v175: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:51 smithi145 ceph-mon[97681]: pgmap v176: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:51 smithi162 ceph-mon[130941]: pgmap v176: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:51 smithi183 ceph-mon[131414]: pgmap v176: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:53 smithi145 ceph-mon[97681]: pgmap v177: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:53 smithi162 ceph-mon[130941]: pgmap v177: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:53 smithi183 ceph-mon[131414]: pgmap v177: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:55 smithi145 ceph-mon[97681]: pgmap v178: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:55 smithi162 ceph-mon[130941]: pgmap v178: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:56.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:55 smithi183 ceph-mon[131414]: pgmap v178: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:50:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:57 smithi145 ceph-mon[97681]: pgmap v179: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:57 smithi162 ceph-mon[130941]: pgmap v179: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:50:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:57 smithi183 ceph-mon[131414]: pgmap v179: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:50:59 smithi145 ceph-mon[97681]: pgmap v180: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:00.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:50:59 smithi162 ceph-mon[130941]: pgmap v180: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:50:59 smithi183 ceph-mon[131414]: pgmap v180: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:02.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:01 smithi145 ceph-mon[97681]: pgmap v181: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:01 smithi162 ceph-mon[130941]: pgmap v181: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:01 smithi183 ceph-mon[131414]: pgmap v181: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:04.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:03 smithi145 ceph-mon[97681]: pgmap v182: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:03 smithi162 ceph-mon[130941]: pgmap v182: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:03 smithi183 ceph-mon[131414]: pgmap v182: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:05.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:51:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:05 smithi145 ceph-mon[97681]: pgmap v183: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:06.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.113+0000 7f04e95ff700 1 -- 172.21.15.145:0/2907170127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4108ad0 msgr2=0x7f04e4108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.113+0000 7f04e95ff700 1 --2- 172.21.15.145:0/2907170127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4108ad0 0x7f04e4108eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009990 tx=0x7f04d4009ca0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 -- 172.21.15.145:0/2907170127 shutdown_connections 2023-08-25T04:51:06.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 --2- 172.21.15.145:0/2907170127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4103480 0x7f04e410f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 --2- 172.21.15.145:0/2907170127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4108ad0 0x7f04e4108eb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 --2- 172.21.15.145:0/2907170127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4102ac0 0x7f04e4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 -- 172.21.15.145:0/2907170127 >> 172.21.15.145:0/2907170127 conn(0x7f04e40fe570 msgr2=0x7f04e4100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 -- 172.21.15.145:0/2907170127 shutdown_connections 2023-08-25T04:51:06.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.114+0000 7f04e95ff700 1 -- 172.21.15.145:0/2907170127 wait complete. 2023-08-25T04:51:06.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 Processor -- start 2023-08-25T04:51:06.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 -- start start 2023-08-25T04:51:06.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4102ac0 0x7f04e419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4108ad0 0x7f04e4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4199120 con 0x7f04e4102ac0 2023-08-25T04:51:06.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.115+0000 7f04e95ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4199290 con 0x7f04e4108ad0 2023-08-25T04:51:06.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e95ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4199450 con 0x7f04e4103480 2023-08-25T04:51:06.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4102ac0 0x7f04e419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60612/0 (socket says 172.21.15.145:60612) 2023-08-25T04:51:06.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 -- 172.21.15.145:0/4147988036 learned_addr learned my addr 172.21.15.145:0/4147988036 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:06.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e37fe700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4108ad0 0x7f04e4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 -- 172.21.15.145:0/4147988036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4108ad0 msgr2=0x7f04e4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4108ad0 0x7f04e4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 -- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4102ac0 msgr2=0x7f04e419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4102ac0 0x7f04e419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 -- 172.21.15.145:0/4147988036 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40095f0 con 0x7f04e4103480 2023-08-25T04:51:06.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.116+0000 7f04e27fc700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f04d800e9a0 tx=0x7f04d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.117+0000 7f04cbfff700 1 -- 172.21.15.145:0/4147988036 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8009790 con 0x7f04e4103480 2023-08-25T04:51:06.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.117+0000 7f04cbfff700 1 -- 172.21.15.145:0/4147988036 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d8010430 con 0x7f04e4103480 2023-08-25T04:51:06.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.117+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e4199730 con 0x7f04e4103480 2023-08-25T04:51:06.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.117+0000 7f04cbfff700 1 -- 172.21.15.145:0/4147988036 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8018680 con 0x7f04e4103480 2023-08-25T04:51:06.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.117+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e41a59e0 con 0x7f04e4103480 2023-08-25T04:51:06.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.121+0000 7f04cbfff700 1 -- 172.21.15.145:0/4147988036 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f04d80105a0 con 0x7f04e4103480 2023-08-25T04:51:06.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.122+0000 7f04fcfe8b80 1 Processor -- start 2023-08-25T04:51:06.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.122+0000 7f04fcfe8b80 1 -- start start 2023-08-25T04:51:06.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.122+0000 7f04cbfff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 0x7f04cc0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.122+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1113d7b0 con 0x55be111dbca0 2023-08-25T04:51:06.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1112e780 con 0x55be111d5c40 2023-08-25T04:51:06.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be1112c0a0 con 0x55be111db380 2023-08-25T04:51:06.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04cbfff700 1 -- 172.21.15.145:0/4147988036 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f04d8014070 con 0x7f04e4103480 2023-08-25T04:51:06.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e37fe700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 0x7f04cc0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60626/0 (socket says 172.21.15.145:60626) 2023-08-25T04:51:06.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 -- 172.21.15.145:0/1934775448 learned_addr learned my addr 172.21.15.145:0/1934775448 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:06.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e27fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e37fe700 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 -- 172.21.15.145:0/1934775448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 msgr2=0x55be111d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 -- 172.21.15.145:0/1934775448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 msgr2=0x55be111d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e2ffd700 1 -- 172.21.15.145:0/1934775448 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be10fa7630 con 0x55be111db380 2023-08-25T04:51:06.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.123+0000 7f04e37fe700 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:51:06.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.124+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005ea0 tx=0x7f04d4005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1642a08a07695e60 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.124+0000 7f04e37fe700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 0x7f04cc0a2600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009ec0 tx=0x7f04dc009920 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.124+0000 7f04c97fa700 1 -- 172.21.15.145:0/1934775448 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d401d070 con 0x55be111db380 2023-08-25T04:51:06.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.124+0000 7f04c97fa700 1 -- 172.21.15.145:0/1934775448 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d401e040 con 0x55be111db380 2023-08-25T04:51:06.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.124+0000 7f04c97fa700 1 -- 172.21.15.145:0/1934775448 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40048f0 con 0x55be111db380 2023-08-25T04:51:06.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/1934775448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 msgr2=0x55be111db760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005ea0 tx=0x7f04d4005ed0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/1934775448 shutdown_connections 2023-08-25T04:51:06.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111db760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/1934775448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/1934775448 >> 172.21.15.145:0/1934775448 conn(0x55be1114a0e0 msgr2=0x55be111d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/1934775448 shutdown_connections 2023-08-25T04:51:06.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.125+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/1934775448 wait complete. 2023-08-25T04:51:06.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 -- start start 2023-08-25T04:51:06.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 Processor -- start 2023-08-25T04:51:06.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111db380 0x55be111dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111dbca0 0x55be111df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be11127990 con 0x55be111db380 2023-08-25T04:51:06.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be110c5bb0 con 0x55be111dbca0 2023-08-25T04:51:06.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.126+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be110c58b0 con 0x55be111d5c40 2023-08-25T04:51:06.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04fcfe8b80 10 client.0 _ll_register_callbacks cb 0x55be1100fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:51:06.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60640/0 (socket says 172.21.15.145:60640) 2023-08-25T04:51:06.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 -- 172.21.15.145:0/2587774778 learned_addr learned my addr 172.21.15.145:0/2587774778 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:06.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e37fe700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111dbca0 0x55be111df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e27fc700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111db380 0x55be111dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 -- 172.21.15.145:0/2587774778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111dbca0 msgr2=0x55be111df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111dbca0 0x55be111df500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 -- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111db380 msgr2=0x55be111dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111db380 0x55be111dee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 -- 172.21.15.145:0/2587774778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e4110290 con 0x55be111d5c40 2023-08-25T04:51:06.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009350 tx=0x7f04d400fb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc140b311fafdcd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.127+0000 7f04c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:51:06.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d401d070 con 0x55be111d5c40 2023-08-25T04:51:06.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be111a51c0 con 0x55be111d5c40 2023-08-25T04:51:06.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f04d4005030 con 0x55be111d5c40 2023-08-25T04:51:06.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d4005610 con 0x55be111d5c40 2023-08-25T04:51:06.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04fcfe8b80 10 client.25960 fetch_fsmap learned FSMap version 34 2023-08-25T04:51:06.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55be10f53de0 con 0x55be111d5c40 2023-08-25T04:51:06.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be111a51c0 con 0x55be111d5c40 2023-08-25T04:51:06.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.128+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f04d4035070 con 0x55be111d5c40 2023-08-25T04:51:06.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 34 2023-08-25T04:51:06.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:51:06.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 20 client.25960 populate_metadata read hostname 'smithi145' 2023-08-25T04:51:06.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be111ef110 0x55be111f1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 4 client.25960 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:51:06.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be110cdba0 con 0x55be111ef110 2023-08-25T04:51:06.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04e27fc700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be111ef110 0x55be111f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04c6ff5700 10 client.25960 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:06.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.129+0000 7f04e27fc700 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be111ef110 0x55be111f1500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02ce6d3271c1804 server_cookie=240e91d660d75224 in_seq=0 out_seq=0 2023-08-25T04:51:06.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.130+0000 7f04c6ff5700 1 -- 172.21.15.145:0/2587774778 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be110cdba0 con 0x55be111ef110 2023-08-25T04:51:06.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.130+0000 7f04c6ff5700 10 client.25960 handle_command_reply: tid=0 2023-08-25T04:51:06.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.130+0000 7f04fcfe8b80 1 client.25960 shutdown 2023-08-25T04:51:06.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.130+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 msgr2=0x55be111e1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.130+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009350 tx=0x7f04d400fb90 comp rx=0 tx=0).stop 2023-08-25T04:51:06.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 shutdown_connections 2023-08-25T04:51:06.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111dbca0 0x55be111df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be111ef110 0x55be111f1500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111e1c00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/2587774778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111db380 0x55be111dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04c6ff5700 0 client.25960 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:06.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 >> 172.21.15.145:0/2587774778 conn(0x55be1114a0e0 msgr2=0x55be111cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 shutdown_connections 2023-08-25T04:51:06.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/2587774778 wait complete. 2023-08-25T04:51:06.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.131+0000 7f04fcfe8b80 20 client.25960 trim_cache size 0 max 16384 2023-08-25T04:51:06.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 Processor -- start 2023-08-25T04:51:06.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 -- start start 2023-08-25T04:51:06.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be10fa9920 con 0x55be111dbca0 2023-08-25T04:51:06.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be111c86f0 con 0x55be111d5c40 2023-08-25T04:51:06.161 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be111db760 con 0x55be111db380 2023-08-25T04:51:06.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04e27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04e27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60656/0 (socket says 172.21.15.145:60656) 2023-08-25T04:51:06.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04e27fc700 1 -- 172.21.15.145:0/3504810608 learned_addr learned my addr 172.21.15.145:0/3504810608 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:06.162 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.151+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e37fe700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 -- 172.21.15.145:0/3504810608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 msgr2=0x55be111e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 -- 172.21.15.145:0/3504810608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 msgr2=0x55be111d94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.163 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 -- 172.21.15.145:0/3504810608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be111f60c0 con 0x55be111db380 2023-08-25T04:51:06.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e37fe700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:51:06.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04e27fc700 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04d8010c30 tx=0x7f04d800ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd354df7f852478d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04c9ffb700 1 -- 172.21.15.145:0/3504810608 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d80995a0 con 0x55be111db380 2023-08-25T04:51:06.164 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04c9ffb700 1 -- 172.21.15.145:0/3504810608 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d80b6e90 con 0x55be111db380 2023-08-25T04:51:06.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.152+0000 7f04c9ffb700 1 -- 172.21.15.145:0/3504810608 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8010880 con 0x55be111db380 2023-08-25T04:51:06.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/3504810608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 msgr2=0x55be111d8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04d8010c30 tx=0x7f04d800ff60 comp rx=0 tx=0).stop 2023-08-25T04:51:06.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/3504810608 shutdown_connections 2023-08-25T04:51:06.165 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111d5c40 0x55be111e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/3504810608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111db380 0x55be111d8fb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/3504810608 >> 172.21.15.145:0/3504810608 conn(0x55be1114a0e0 msgr2=0x55be111eb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.166 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/3504810608 shutdown_connections 2023-08-25T04:51:06.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/3504810608 wait complete. 2023-08-25T04:51:06.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.153+0000 7f04fcfe8b80 1 -- start start 2023-08-25T04:51:06.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 Processor -- start 2023-08-25T04:51:06.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.167 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111db380 0x55be111d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be111d73d0 con 0x55be111dbca0 2023-08-25T04:51:06.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be111d7510 con 0x55be111db380 2023-08-25T04:51:06.168 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be111d76d0 con 0x55be111d5c40 2023-08-25T04:51:06.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04fcfe8b80 10 client.0 _ll_register_callbacks cb 0x7f04d8010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:51:06.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60666/0 (socket says 172.21.15.145:60666) 2023-08-25T04:51:06.169 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.154+0000 7f04e2ffd700 1 -- 172.21.15.145:0/288529640 learned_addr learned my addr 172.21.15.145:0/288529640 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:06.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e37fe700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e27fc700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111db380 0x55be111d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.170 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 -- 172.21.15.145:0/288529640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111db380 msgr2=0x55be111d86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111db380 0x55be111d86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 -- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 msgr2=0x55be111d6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:06.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 -- 172.21.15.145:0/288529640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e41a59e0 con 0x55be111d5c40 2023-08-25T04:51:06.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e37fe700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:51:06.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04e2ffd700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eafe43b60a3d36c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:06.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:51:06.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d401d070 con 0x55be111d5c40 2023-08-25T04:51:06.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f04d4000e40 con 0x55be111d5c40 2023-08-25T04:51:06.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.155+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be111c88b0 con 0x55be111d5c40 2023-08-25T04:51:06.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d4005bc0 con 0x55be111d5c40 2023-08-25T04:51:06.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 10 client.25966 fetch_fsmap learned FSMap version 34 2023-08-25T04:51:06.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55be10f3cc60 con 0x55be111d5c40 2023-08-25T04:51:06.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be111c88b0 con 0x55be111d5c40 2023-08-25T04:51:06.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f04d400b850 con 0x55be111d5c40 2023-08-25T04:51:06.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 34 2023-08-25T04:51:06.174 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 20 client.25966 populate_metadata read hostname 'smithi145' 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be11243a70 0x55be11245e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 4 client.25966 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be11120970 con 0x55be11243a70 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.156+0000 7f04e27fc700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be11243a70 0x55be11245e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:06.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04c6ff5700 10 client.25966 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:06.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04e27fc700 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be11243a70 0x55be11245e60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821f407f1b0bf2ba server_cookie=9645be2120763e39 in_seq=0 out_seq=0 2023-08-25T04:51:06.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04c6ff5700 1 -- 172.21.15.145:0/288529640 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55be11120970 con 0x55be11243a70 2023-08-25T04:51:06.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04c6ff5700 10 client.25966 handle_command_reply: tid=0 2023-08-25T04:51:06.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 client.25966 shutdown 2023-08-25T04:51:06.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 msgr2=0x55be111d8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d4004790 comp rx=0 tx=0).stop 2023-08-25T04:51:06.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 shutdown_connections 2023-08-25T04:51:06.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55be111dbca0 0x55be111d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.157+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55be11243a70 0x55be11245e60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be111d5c40 0x55be111d8180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04c6ff5700 0 client.25966 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:06.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 1 --2- 172.21.15.145:0/288529640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be111db380 0x55be111d86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 >> 172.21.15.145:0/288529640 conn(0x55be1114a0e0 msgr2=0x55be111d8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 shutdown_connections 2023-08-25T04:51:06.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 1 -- 172.21.15.145:0/288529640 wait complete. 2023-08-25T04:51:06.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04fcfe8b80 20 client.25966 trim_cache size 0 max 16384 2023-08-25T04:51:06.180 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:51:06.181 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:51:06.181 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:51:06.181 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:51:06.181 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:51:06.182 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:51:06.182 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:51:06.182 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:51:06.182 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:51:06.183 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:51:06.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 msgr2=0x7f04cc0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 0x7f04cc0a2600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009ec0 tx=0x7f04dc009920 comp rx=0 tx=0).stop 2023-08-25T04:51:06.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.158+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 msgr2=0x7f04e419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:06.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f04d800e9a0 tx=0x7f04d800ecb0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 shutdown_connections 2023-08-25T04:51:06.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f04cc0a0140 0x7f04cc0a2600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04e4108ad0 0x7f04e4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.185 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4102ac0 0x7f04e419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 --2- 172.21.15.145:0/4147988036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e4103480 0x7f04e419ef30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:06.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 >> 172.21.15.145:0/4147988036 conn(0x7f04e40fe570 msgr2=0x7f04e40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:06.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 shutdown_connections 2023-08-25T04:51:06.186 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:06.159+0000 7f04e95ff700 1 -- 172.21.15.145:0/4147988036 wait complete. 2023-08-25T04:51:06.187 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:51:06.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:05 smithi183 ceph-mon[131414]: pgmap v183: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:06.189 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:05 smithi162 ceph-mon[130941]: pgmap v183: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:08.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:07 smithi145 ceph-mon[97681]: pgmap v184: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:07 smithi162 ceph-mon[130941]: pgmap v184: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:07 smithi183 ceph-mon[131414]: pgmap v184: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-08-25T04:51:10.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:09 smithi145 ceph-mon[97681]: pgmap v185: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:09 smithi162 ceph-mon[130941]: pgmap v185: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:09 smithi183 ceph-mon[131414]: pgmap v185: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-08-25T04:51:11.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:10 smithi145 ceph-mon[97681]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:51:11.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:10 smithi162 ceph-mon[130941]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:51:11.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:10 smithi183 ceph-mon[131414]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-08-25T04:51:12.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:11 smithi145 ceph-mon[97681]: pgmap v186: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:51:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:11 smithi162 ceph-mon[130941]: pgmap v186: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:51:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:11 smithi183 ceph-mon[131414]: pgmap v186: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-08-25T04:51:14.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:13 smithi145 ceph-mon[97681]: pgmap v187: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:14.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:13 smithi162 ceph-mon[130941]: pgmap v187: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:13 smithi183 ceph-mon[131414]: pgmap v187: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:15 smithi145 ceph-mon[97681]: pgmap v188: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:15 smithi162 ceph-mon[130941]: pgmap v188: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:15 smithi183 ceph-mon[131414]: pgmap v188: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-08-25T04:51:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:17 smithi145 ceph-mon[97681]: pgmap v189: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:51:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:17 smithi162 ceph-mon[130941]: pgmap v189: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:51:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:17 smithi183 ceph-mon[131414]: pgmap v189: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2023-08-25T04:51:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:19 smithi145 ceph-mon[97681]: pgmap v190: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:51:20.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:19 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:51:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:19 smithi162 ceph-mon[130941]: pgmap v190: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:51:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:19 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:51:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:19 smithi183 ceph-mon[131414]: pgmap v190: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-08-25T04:51:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:19 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:51:21.081 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:20 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:51:21.082 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:20 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:51:21.082 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:20 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:51:21.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:20 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:51:21.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:20 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:51:21.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:20 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:51:21.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:20 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:51:21.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:20 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:51:21.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:20 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:51:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:21 smithi145 ceph-mon[97681]: pgmap v191: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:51:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:21 smithi162 ceph-mon[130941]: pgmap v191: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:51:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:21 smithi183 ceph-mon[131414]: pgmap v191: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-08-25T04:51:24.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:23 smithi145 ceph-mon[97681]: pgmap v192: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:23 smithi162 ceph-mon[130941]: pgmap v192: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:24.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:23 smithi183 ceph-mon[131414]: pgmap v192: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:26.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:25 smithi145 ceph-mon[97681]: pgmap v193: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:25 smithi162 ceph-mon[130941]: pgmap v193: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:25 smithi183 ceph-mon[131414]: pgmap v193: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:28.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:27 smithi145 ceph-mon[97681]: pgmap v194: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:27 smithi162 ceph-mon[130941]: pgmap v194: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:27 smithi183 ceph-mon[131414]: pgmap v194: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:29 smithi145 ceph-mon[97681]: pgmap v195: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:29 smithi162 ceph-mon[130941]: pgmap v195: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:29 smithi183 ceph-mon[131414]: pgmap v195: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:31 smithi145 ceph-mon[97681]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:31 smithi162 ceph-mon[130941]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:31 smithi183 ceph-mon[131414]: pgmap v196: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:33 smithi145 ceph-mon[97681]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:33 smithi162 ceph-mon[130941]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:33 smithi183 ceph-mon[131414]: pgmap v197: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:35 smithi145 ceph-mon[97681]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:35 smithi162 ceph-mon[130941]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:35 smithi183 ceph-mon[131414]: pgmap v198: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:36.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:51:36.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 -- 172.21.15.145:0/2658913660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 msgr2=0x7f650810d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.421 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 --2- 172.21.15.145:0/2658913660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650810d320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f650000b0a0 tx=0x7f650000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 -- 172.21.15.145:0/2658913660 shutdown_connections 2023-08-25T04:51:36.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 --2- 172.21.15.145:0/2658913660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650810d320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 --2- 172.21.15.145:0/2658913660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 --2- 172.21.15.145:0/2658913660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 0x7f650810a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.422 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 -- 172.21.15.145:0/2658913660 >> 172.21.15.145:0/2658913660 conn(0x7f65080fb260 msgr2=0x7f65080fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 -- 172.21.15.145:0/2658913660 shutdown_connections 2023-08-25T04:51:36.423 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.420+0000 7f650edff700 1 -- 172.21.15.145:0/2658913660 wait complete. 2023-08-25T04:51:36.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.421+0000 7f650edff700 1 Processor -- start 2023-08-25T04:51:36.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.421+0000 7f650edff700 1 -- start start 2023-08-25T04:51:36.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 0x7f6508199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.427 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819cbb0 con 0x7f6508101a40 2023-08-25T04:51:36.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819cd20 con 0x7f6508102360 2023-08-25T04:51:36.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650edff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819cee0 con 0x7f650810ae60 2023-08-25T04:51:36.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45274/0 (socket says 172.21.15.145:45274) 2023-08-25T04:51:36.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 -- 172.21.15.145:0/2041583107 learned_addr learned my addr 172.21.15.145:0/2041583107 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:36.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f6507fff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 0x7f6508199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650d39c700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 -- 172.21.15.145:0/2041583107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 msgr2=0x7f650819c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650819c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 -- 172.21.15.145:0/2041583107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 msgr2=0x7f6508199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 0x7f6508199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.422+0000 7f650cb9b700 1 -- 172.21.15.145:0/2041583107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009d00 con 0x7f6508101a40 2023-08-25T04:51:36.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f650cb9b700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9a0 tx=0x7f64fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f6505ffb700 1 -- 172.21.15.145:0/2041583107 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc009790 con 0x7f6508101a40 2023-08-25T04:51:36.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650819d1c0 con 0x7f6508101a40 2023-08-25T04:51:36.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f6505ffb700 1 -- 172.21.15.145:0/2041583107 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64fc010430 con 0x7f6508101a40 2023-08-25T04:51:36.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f6505ffb700 1 -- 172.21.15.145:0/2041583107 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc018680 con 0x7f6508101a40 2023-08-25T04:51:36.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.423+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65081a5b20 con 0x7f6508101a40 2023-08-25T04:51:36.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.427+0000 7f6505ffb700 1 -- 172.21.15.145:0/2041583107 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f64fc0105a0 con 0x7f6508101a40 2023-08-25T04:51:36.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.428+0000 7f6522699b80 1 Processor -- start 2023-08-25T04:51:36.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.428+0000 7f6522699b80 1 -- start start 2023-08-25T04:51:36.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59e3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d594b7b0 con 0x55a8d59e9380 2023-08-25T04:51:36.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d593c780 con 0x55a8d59e3c40 2023-08-25T04:51:36.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6522699b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d593a0a0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6505ffb700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 0x7f64f80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f650d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6507fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f650d39c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59e9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45288/0 (socket says 172.21.15.145:45288) 2023-08-25T04:51:36.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f650d39c700 1 -- 172.21.15.145:0/1073219307 learned_addr learned my addr 172.21.15.145:0/1073219307 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:36.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f6505ffb700 1 -- 172.21.15.145:0/2041583107 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f64fc014070 con 0x7f6508101a40 2023-08-25T04:51:36.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.429+0000 7f650d39c700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 0x7f64f80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 -- 172.21.15.145:0/1073219307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 msgr2=0x55a8d59e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 -- 172.21.15.145:0/1073219307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 msgr2=0x55a8d59e9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59e9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 -- 172.21.15.145:0/1073219307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d57b5630 con 0x55a8d59e9ca0 2023-08-25T04:51:36.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59e3700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0c8ee0 tx=0x7f64fc064930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82c10f6696b238cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f64f2ffd700 1 -- 172.21.15.145:0/1073219307 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01ce40 con 0x55a8d59e9ca0 2023-08-25T04:51:36.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f650d39c700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 0x7f64f80a2600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650000b0a0 tx=0x7f6500006230 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f64f2ffd700 1 -- 172.21.15.145:0/1073219307 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64fc0b6e90 con 0x55a8d59e9ca0 2023-08-25T04:51:36.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.430+0000 7f64f2ffd700 1 -- 172.21.15.145:0/1073219307 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0c9160 con 0x55a8d59e9ca0 2023-08-25T04:51:36.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 -- 172.21.15.145:0/1073219307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 msgr2=0x55a8d59e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59e3700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0c8ee0 tx=0x7f64fc064930 comp rx=0 tx=0).stop 2023-08-25T04:51:36.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 -- 172.21.15.145:0/1073219307 shutdown_connections 2023-08-25T04:51:36.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59e9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 --2- 172.21.15.145:0/1073219307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59e3700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 -- 172.21.15.145:0/1073219307 >> 172.21.15.145:0/1073219307 conn(0x55a8d59580e0 msgr2=0x55a8d59de410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 -- 172.21.15.145:0/1073219307 shutdown_connections 2023-08-25T04:51:36.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.431+0000 7f6522699b80 1 -- 172.21.15.145:0/1073219307 wait complete. 2023-08-25T04:51:36.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 -- start start 2023-08-25T04:51:36.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 Processor -- start 2023-08-25T04:51:36.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e3c40 0x55a8d59efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d5935990 con 0x55a8d59e3c40 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d58d3bb0 con 0x55a8d59e9380 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.432+0000 7f6522699b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d58d38b0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f6522699b80 10 client.0 _ll_register_callbacks cb 0x55a8d581da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:51:36.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e3c40 0x55a8d59efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54686/0 (socket says 172.21.15.145:54686) 2023-08-25T04:51:36.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 -- 172.21.15.145:0/1003633177 learned_addr learned my addr 172.21.15.145:0/1003633177 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:36.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f6507fff700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 -- 172.21.15.145:0/1003633177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 msgr2=0x55a8d59eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 -- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e3c40 msgr2=0x55a8d59efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e3c40 0x55a8d59efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.433+0000 7f650d39c700 1 -- 172.21.15.145:0/1003633177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65081a5b20 con 0x55a8d59e9ca0 2023-08-25T04:51:36.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f650d39c700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6500003c60 tx=0x7f65000099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee61cecc751c3345 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f64e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:51:36.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650000e040 con 0x55a8d59e9ca0 2023-08-25T04:51:36.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8d59b31c0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f6500020450 con 0x55a8d59e9ca0 2023-08-25T04:51:36.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6500034640 con 0x55a8d59e9ca0 2023-08-25T04:51:36.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f6522699b80 10 client.25978 fetch_fsmap learned FSMap version 35 2023-08-25T04:51:36.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a8d5761de0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.434+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8d59b31c0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f650002a070 con 0x55a8d59e9ca0 2023-08-25T04:51:36.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:51:36.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:51:36.460 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 20 client.25978 populate_metadata read hostname 'smithi145' 2023-08-25T04:51:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d59fd6d0 0x55a8d59ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 4 client.25978 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:51:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8d58dbba0 con 0x55a8d59fd6d0 2023-08-25T04:51:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.435+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d59fd6d0 0x55a8d59ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.436+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d59fd6d0 0x55a8d59ffaf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f118ab8750ee45a server_cookie=4a82242ca2228d63 in_seq=0 out_seq=0 2023-08-25T04:51:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.436+0000 7f64e7fff700 10 client.25978 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:36.463 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.436+0000 7f64e7fff700 1 -- 172.21.15.145:0/1003633177 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8d58dbba0 con 0x55a8d59fd6d0 2023-08-25T04:51:36.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.436+0000 7f64e7fff700 10 client.25978 handle_command_reply: tid=0 2023-08-25T04:51:36.464 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.436+0000 7f6522699b80 1 client.25978 shutdown 2023-08-25T04:51:36.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 msgr2=0x55a8d59ed610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6500003c60 tx=0x7f65000099e0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.466 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 shutdown_connections 2023-08-25T04:51:36.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d59fd6d0 0x55a8d59ffaf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ed610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e3c40 0x55a8d59efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.467 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f64e7fff700 0 client.25978 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:36.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 --2- 172.21.15.145:0/1003633177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 >> 172.21.15.145:0/1003633177 conn(0x55a8d59580e0 msgr2=0x55a8d59d82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 shutdown_connections 2023-08-25T04:51:36.468 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 1 -- 172.21.15.145:0/1003633177 wait complete. 2023-08-25T04:51:36.469 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.437+0000 7f6522699b80 20 client.25978 trim_cache size 0 max 16384 2023-08-25T04:51:36.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.465+0000 7f6522699b80 1 Processor -- start 2023-08-25T04:51:36.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 -- start start 2023-08-25T04:51:36.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.473 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d57b7920 con 0x55a8d59e9ca0 2023-08-25T04:51:36.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d59dac80 con 0x55a8d59e9380 2023-08-25T04:51:36.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6522699b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d59dfd40 con 0x55a8d59e3c40 2023-08-25T04:51:36.474 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54692/0 (socket says 172.21.15.145:54692) 2023-08-25T04:51:36.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f650cb9b700 1 -- 172.21.15.145:0/3256995236 learned_addr learned my addr 172.21.15.145:0/3256995236 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:36.475 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f650d39c700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.477 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.466+0000 7f6507fff700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.478 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 -- 172.21.15.145:0/3256995236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 msgr2=0x55a8d59ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 -- 172.21.15.145:0/3256995236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 msgr2=0x55a8d59ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.479 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 -- 172.21.15.145:0/3256995236 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d574ac60 con 0x55a8d59e3c40 2023-08-25T04:51:36.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650d39c700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:51:36.480 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f650cb9b700 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0971f0 tx=0x7f64fc064dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f96aca19c000b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.481 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f64f37fe700 1 -- 172.21.15.145:0/3256995236 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc098f20 con 0x55a8d59e3c40 2023-08-25T04:51:36.483 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f64f37fe700 1 -- 172.21.15.145:0/3256995236 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64fc0b1030 con 0x55a8d59e3c40 2023-08-25T04:51:36.484 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.467+0000 7f64f37fe700 1 -- 172.21.15.145:0/3256995236 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc099100 con 0x55a8d59e3c40 2023-08-25T04:51:36.484 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- 172.21.15.145:0/3256995236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 msgr2=0x55a8d59e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.484 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0971f0 tx=0x7f64fc064dd0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.484 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- 172.21.15.145:0/3256995236 shutdown_connections 2023-08-25T04:51:36.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9ca0 0x55a8d59ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e3c40 0x55a8d59e3700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 --2- 172.21.15.145:0/3256995236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e9380 0x55a8d59ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- 172.21.15.145:0/3256995236 >> 172.21.15.145:0/3256995236 conn(0x55a8d59580e0 msgr2=0x55a8d59e7050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.485 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- 172.21.15.145:0/3256995236 shutdown_connections 2023-08-25T04:51:36.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- 172.21.15.145:0/3256995236 wait complete. 2023-08-25T04:51:36.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.468+0000 7f6522699b80 1 -- start start 2023-08-25T04:51:36.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 Processor -- start 2023-08-25T04:51:36.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.486 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d59ed070 con 0x55a8d59e9380 2023-08-25T04:51:36.487 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d59ed1e0 con 0x55a8d59e3c40 2023-08-25T04:51:36.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d59ebf00 con 0x55a8d59e9ca0 2023-08-25T04:51:36.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6522699b80 10 client.0 _ll_register_callbacks cb 0x55a8d581da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:51:36.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f650d39c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f6507fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f650d39c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:54708/0 (socket says 172.21.15.145:54708) 2023-08-25T04:51:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f650d39c700 1 -- 172.21.15.145:0/1327583883 learned_addr learned my addr 172.21.15.145:0/1327583883 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:51:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.469+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 -- 172.21.15.145:0/1327583883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 msgr2=0x55a8d59ec9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59ec9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 -- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 msgr2=0x55a8d59eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:51:36.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 -- 172.21.15.145:0/1327583883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8d57b5630 con 0x55a8d59e9ca0 2023-08-25T04:51:36.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f650d39c700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6500009cb0 tx=0x7f6500008fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4fedae51bb12ba1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:51:36.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f64e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:51:36.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650002dbb0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8d59d68b0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.470+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9257+0+0 (secure 0 0 0) 0x7f650002dd10 con 0x55a8d59e9ca0 2023-08-25T04:51:36.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65000123f0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 10 client.25984 fetch_fsmap learned FSMap version 35 2023-08-25T04:51:36.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a8d5788250 con 0x55a8d59e9ca0 2023-08-25T04:51:36.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8d59d68b0 con 0x55a8d59e9ca0 2023-08-25T04:51:36.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f650003b070 con 0x55a8d59e9ca0 2023-08-25T04:51:36.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 10 client.25984 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:51:36.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:51:36.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 20 client.25984 populate_metadata read hostname 'smithi145' 2023-08-25T04:51:36.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d5a523a0 0x55a8d5a54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:51:36.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 4 client.25984 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:51:36.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8d592e970 con 0x55a8d5a523a0 2023-08-25T04:51:36.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.471+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d5a523a0 0x55a8d5a54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:51:36.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f650cb9b700 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d5a523a0 0x55a8d5a54790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a47b2d0c93c069e server_cookie=5e29ef37ad8a3560 in_seq=0 out_seq=0 2023-08-25T04:51:36.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f64e7fff700 10 client.25984 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:36.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f64e7fff700 1 -- 172.21.15.145:0/1327583883 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a8d592e970 con 0x55a8d5a523a0 2023-08-25T04:51:36.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f64e7fff700 10 client.25984 handle_command_reply: tid=0 2023-08-25T04:51:36.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 client.25984 shutdown 2023-08-25T04:51:36.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 msgr2=0x55a8d59eb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6500009cb0 tx=0x7f6500008fa0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 shutdown_connections 2023-08-25T04:51:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a8d5a523a0 0x55a8d5a54790 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d59e9ca0 0x55a8d59eb9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8d59e3c40 0x55a8d59ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 --2- 172.21.15.145:0/1327583883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a8d59e9380 0x55a8d59eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.472+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 >> 172.21.15.145:0/1327583883 conn(0x55a8d59580e0 msgr2=0x55a8d59e7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.499 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f64e7fff700 0 client.25984 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:51:36.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 shutdown_connections 2023-08-25T04:51:36.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f6522699b80 1 -- 172.21.15.145:0/1327583883 wait complete. 2023-08-25T04:51:36.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f6522699b80 20 client.25984 trim_cache size 0 max 16384 2023-08-25T04:51:36.501 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:51:36.501 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:51:36.501 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:51:36.501 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:51:36.502 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:51:36.502 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:51:36.502 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:51:36.502 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:51:36.503 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:51:36.503 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:51:36.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 msgr2=0x7f64f80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 0x7f64f80a2600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f650000b0a0 tx=0x7f6500006230 comp rx=0 tx=0).stop 2023-08-25T04:51:36.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 msgr2=0x7f6508198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:51:36.505 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9a0 tx=0x7f64fc00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 shutdown_connections 2023-08-25T04:51:36.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f650810ae60 0x7f650819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6508101a40 0x7f6508198cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f64f80a0140 0x7f64f80a2600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.506 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.473+0000 7f650edff700 1 --2- 172.21.15.145:0/2041583107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6508102360 0x7f6508199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:51:36.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.474+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 >> 172.21.15.145:0/2041583107 conn(0x7f65080fb260 msgr2=0x7f650810cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:51:36.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.474+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 shutdown_connections 2023-08-25T04:51:36.507 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:51:36.474+0000 7f650edff700 1 -- 172.21.15.145:0/2041583107 wait complete. 2023-08-25T04:51:36.508 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:51:38.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:37 smithi145 ceph-mon[97681]: pgmap v199: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:37 smithi162 ceph-mon[130941]: pgmap v199: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:37 smithi183 ceph-mon[131414]: pgmap v199: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:40.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:39 smithi145 ceph-mon[97681]: pgmap v200: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:39 smithi162 ceph-mon[130941]: pgmap v200: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:39 smithi183 ceph-mon[131414]: pgmap v200: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:42.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:41 smithi145 ceph-mon[97681]: pgmap v201: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:42.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:41 smithi183 ceph-mon[131414]: pgmap v201: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:42.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:41 smithi162 ceph-mon[130941]: pgmap v201: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:44.080 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:43 smithi145 ceph-mon[97681]: pgmap v202: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:43 smithi162 ceph-mon[130941]: pgmap v202: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:43 smithi183 ceph-mon[131414]: pgmap v202: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:46.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:45 smithi145 ceph-mon[97681]: pgmap v203: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:45 smithi162 ceph-mon[130941]: pgmap v203: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:45 smithi183 ceph-mon[131414]: pgmap v203: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:51:48.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:47 smithi145 ceph-mon[97681]: pgmap v204: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:47 smithi162 ceph-mon[130941]: pgmap v204: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:47 smithi183 ceph-mon[131414]: pgmap v204: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:49 smithi145 ceph-mon[97681]: pgmap v205: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:49 smithi162 ceph-mon[130941]: pgmap v205: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:49 smithi183 ceph-mon[131414]: pgmap v205: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:51 smithi145 ceph-mon[97681]: pgmap v206: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:52.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:51 smithi162 ceph-mon[130941]: pgmap v206: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:51 smithi183 ceph-mon[131414]: pgmap v206: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:51:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:53 smithi145 ceph-mon[97681]: pgmap v207: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:53 smithi162 ceph-mon[130941]: pgmap v207: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:53 smithi183 ceph-mon[131414]: pgmap v207: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:56.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:55 smithi145 ceph-mon[97681]: pgmap v208: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:55 smithi162 ceph-mon[130941]: pgmap v208: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:55 smithi183 ceph-mon[131414]: pgmap v208: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:51:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:57 smithi145 ceph-mon[97681]: pgmap v209: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-08-25T04:51:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:57 smithi162 ceph-mon[130941]: pgmap v209: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-08-25T04:51:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:57 smithi183 ceph-mon[131414]: pgmap v209: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2023-08-25T04:52:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:51:59 smithi145 ceph-mon[97681]: pgmap v210: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:52:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:51:59 smithi162 ceph-mon[130941]: pgmap v210: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:52:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:51:59 smithi183 ceph-mon[131414]: pgmap v210: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-08-25T04:52:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:01 smithi162 ceph-mon[130941]: pgmap v211: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-08-25T04:52:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:01 smithi183 ceph-mon[131414]: pgmap v211: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-08-25T04:52:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:01 smithi145 ceph-mon[97681]: pgmap v211: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-08-25T04:52:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:03 smithi162 ceph-mon[130941]: pgmap v212: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:03 smithi183 ceph-mon[131414]: pgmap v212: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:04.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:03 smithi145 ceph-mon[97681]: pgmap v212: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:05 smithi162 ceph-mon[130941]: pgmap v213: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:05 smithi183 ceph-mon[131414]: pgmap v213: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:05 smithi145 ceph-mon[97681]: pgmap v213: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-08-25T04:52:06.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:52:06.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 -- 172.21.15.145:0/510419522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8067a80 msgr2=0x7f5ff8067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 --2- 172.21.15.145:0/510419522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8067a80 0x7f5ff8067f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009990 tx=0x7f5fec009ca0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 -- 172.21.15.145:0/510419522 shutdown_connections 2023-08-25T04:52:06.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 --2- 172.21.15.145:0/510419522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 --2- 172.21.15.145:0/510419522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8106780 0x7f5ff8106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 --2- 172.21.15.145:0/510419522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8067a80 0x7f5ff8067f00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 -- 172.21.15.145:0/510419522 >> 172.21.15.145:0/510419522 conn(0x7f5ff80750e0 msgr2=0x7f5ff80754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 -- 172.21.15.145:0/510419522 shutdown_connections 2023-08-25T04:52:06.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.738+0000 7f5ff759e700 1 -- 172.21.15.145:0/510419522 wait complete. 2023-08-25T04:52:06.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.739+0000 7f5ff759e700 1 Processor -- start 2023-08-25T04:52:06.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 -- start start 2023-08-25T04:52:06.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8067a80 0x7f5ff81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819b610 con 0x7f5ff8068440 2023-08-25T04:52:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819b780 con 0x7f5ff8067a80 2023-08-25T04:52:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff759e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819b940 con 0x7f5ff8106780 2023-08-25T04:52:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff6d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.740+0000 7f5ff5d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:44844/0 (socket says 172.21.15.145:44844) 2023-08-25T04:52:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff5d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff8196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:39672/0 (socket says 172.21.15.145:39672) 2023-08-25T04:52:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/1759630129 learned_addr learned my addr 172.21.15.145:0/1759630129 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8067a80 0x7f5ff81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/1759630129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8067a80 msgr2=0x7f5ff81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8067a80 0x7f5ff81966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 msgr2=0x7f5ff8196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff8196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/1759630129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0095f0 con 0x7f5ff8106780 2023-08-25T04:52:06.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000ef30 tx=0x7f5ff000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.741+0000 7f5fe77fe700 1 -- 172.21.15.145:0/1759630129 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0018070 con 0x7f5ff8106780 2023-08-25T04:52:06.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.742+0000 7f5fe77fe700 1 -- 172.21.15.145:0/1759630129 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff000f040 con 0x7f5ff8106780 2023-08-25T04:52:06.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.742+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff819bbc0 con 0x7f5ff8106780 2023-08-25T04:52:06.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.742+0000 7f5fe77fe700 1 -- 172.21.15.145:0/1759630129 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000cd00 con 0x7f5ff8106780 2023-08-25T04:52:06.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.742+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81a3460 con 0x7f5ff8106780 2023-08-25T04:52:06.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.745+0000 7f5fe77fe700 1 -- 172.21.15.145:0/1759630129 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f5ff0003a00 con 0x7f5ff8106780 2023-08-25T04:52:06.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.748+0000 7f5fe77fe700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 0x7f5fe00a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.748+0000 7f5fe77fe700 1 -- 172.21.15.145:0/1759630129 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9433+0+0 (secure 0 0 0) 0x7f5ff00cb530 con 0x7f5ff8106780 2023-08-25T04:52:06.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.748+0000 7f601075db80 1 Processor -- start 2023-08-25T04:52:06.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.748+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 0x7f5fe00a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.748+0000 7f601075db80 1 -- start start 2023-08-25T04:52:06.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 0x5592e1859d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 0x7f5fe00a26a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 0x5592e1854490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e17b77b0 con 0x5592e1854050 2023-08-25T04:52:06.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e17a8780 con 0x5592e1859940 2023-08-25T04:52:06.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f601075db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e17a60a0 con 0x5592e185a260 2023-08-25T04:52:06.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff5d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff6d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 0x5592e1854490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff5d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:44850/0 (socket says 172.21.15.145:44850) 2023-08-25T04:52:06.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff5d9b700 1 -- 172.21.15.145:0/11963545 learned_addr learned my addr 172.21.15.145:0/11963545 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:06.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff659c700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 0x5592e1859d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff659c700 1 -- 172.21.15.145:0/11963545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 msgr2=0x5592e1853b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff659c700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.749+0000 7f5ff659c700 1 -- 172.21.15.145:0/11963545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 msgr2=0x5592e1854490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5ff659c700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 0x5592e1854490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5ff659c700 1 -- 172.21.15.145:0/11963545 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e1621630 con 0x5592e1859940 2023-08-25T04:52:06.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:52:06.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 0x5592e1854490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:52:06.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5ff659c700 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 0x5592e1859d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800e3b0 tx=0x7f5fe8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462174ea6480d82c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5fe4ff9700 1 -- 172.21.15.145:0/11963545 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800ed50 con 0x5592e1859940 2023-08-25T04:52:06.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5fe4ff9700 1 -- 172.21.15.145:0/11963545 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe8021070 con 0x5592e1859940 2023-08-25T04:52:06.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.750+0000 7f5fe4ff9700 1 -- 172.21.15.145:0/11963545 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8003ea0 con 0x5592e1859940 2023-08-25T04:52:06.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 -- 172.21.15.145:0/11963545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 msgr2=0x5592e1859d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 0x5592e1859d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800e3b0 tx=0x7f5fe8014000 comp rx=0 tx=0).stop 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 -- 172.21.15.145:0/11963545 shutdown_connections 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1854050 0x5592e1854490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1859940 0x5592e1859d20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 --2- 172.21.15.145:0/11963545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e1853b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 -- 172.21.15.145:0/11963545 >> 172.21.15.145:0/11963545 conn(0x5592e17c40e0 msgr2=0x5592e184a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 -- 172.21.15.145:0/11963545 shutdown_connections 2023-08-25T04:52:06.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.751+0000 7f601075db80 1 -- 172.21.15.145:0/11963545 wait complete. 2023-08-25T04:52:06.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.752+0000 7f601075db80 1 -- start start 2023-08-25T04:52:06.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.752+0000 7f601075db80 1 Processor -- start 2023-08-25T04:52:06.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e185d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e185dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e17a1990 con 0x5592e1859940 2023-08-25T04:52:06.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e173fbb0 con 0x5592e185a260 2023-08-25T04:52:06.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e173f8b0 con 0x5592e1854050 2023-08-25T04:52:06.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f601075db80 10 client.0 _ll_register_callbacks cb 0x5592e1689a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:52:06.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:44862/0 (socket says 172.21.15.145:44862) 2023-08-25T04:52:06.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 -- 172.21.15.145:0/1638764394 learned_addr learned my addr 172.21.15.145:0/1638764394 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:06.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e185dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e185d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 -- 172.21.15.145:0/1638764394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 msgr2=0x5592e185dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e185dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 -- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 msgr2=0x5592e185d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e185d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.753+0000 7f5ff659c700 1 -- 172.21.15.145:0/1638764394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff810de30 con 0x5592e1854050 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8003a20 tx=0x7f5fe8003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5106460921a8334f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f5fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80052c0 con 0x5592e1854050 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9433+0+0 (secure 0 0 0) 0x7f5fe8003e10 con 0x5592e1854050 2023-08-25T04:52:06.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592e181f1c0 con 0x5592e1854050 2023-08-25T04:52:06.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe8005420 con 0x5592e1854050 2023-08-25T04:52:06.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f601075db80 10 client.25996 fetch_fsmap learned FSMap version 35 2023-08-25T04:52:06.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.754+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f5fe8017070 con 0x5592e1854050 2023-08-25T04:52:06.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592e181f1c0 con 0x5592e1854050 2023-08-25T04:52:06.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5fe802d030 con 0x5592e1854050 2023-08-25T04:52:06.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 10 client.25996 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:52:06.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:52:06.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 20 client.25996 populate_metadata read hostname 'smithi145' 2023-08-25T04:52:06.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e186dba0 0x5592e186ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 4 client.25996 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:52:06.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592e1747ba0 con 0x5592e186dba0 2023-08-25T04:52:06.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.755+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e186dba0 0x5592e186ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.756+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e186dba0 0x5592e186ffc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0bf2e2cebe7227f server_cookie=7e352b19b07058d in_seq=0 out_seq=0 2023-08-25T04:52:06.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.756+0000 7f5fddffb700 10 client.25996 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:06.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.756+0000 7f5fddffb700 1 -- 172.21.15.145:0/1638764394 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5592e1747ba0 con 0x5592e186dba0 2023-08-25T04:52:06.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.756+0000 7f5fddffb700 10 client.25996 handle_command_reply: tid=0 2023-08-25T04:52:06.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.756+0000 7f601075db80 1 client.25996 shutdown 2023-08-25T04:52:06.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 msgr2=0x5592e1860240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8003a20 tx=0x7f5fe8003880 comp rx=0 tx=0).stop 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 shutdown_connections 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e185dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e186dba0 0x5592e186ffc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e1860240 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 --2- 172.21.15.145:0/1638764394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e185d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f5fddffb700 0 client.25996 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:06.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 >> 172.21.15.145:0/1638764394 conn(0x5592e17c40e0 msgr2=0x5592e1852b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 shutdown_connections 2023-08-25T04:52:06.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 1 -- 172.21.15.145:0/1638764394 wait complete. 2023-08-25T04:52:06.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.757+0000 7f601075db80 20 client.25996 trim_cache size 0 max 16384 2023-08-25T04:52:06.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 Processor -- start 2023-08-25T04:52:06.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 -- start start 2023-08-25T04:52:06.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e1864de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e1857480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e186a740 con 0x5592e1859940 2023-08-25T04:52:06.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1623920 con 0x5592e185a260 2023-08-25T04:52:06.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.776+0000 7f601075db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e18548d0 con 0x5592e1854050 2023-08-25T04:52:06.780 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:44870/0 (socket says 172.21.15.145:44870) 2023-08-25T04:52:06.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 -- 172.21.15.145:0/1376281740 learned_addr learned my addr 172.21.15.145:0/1376281740 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:06.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e1864de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e1857480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.781 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 -- 172.21.15.145:0/1376281740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 msgr2=0x5592e1857480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e1857480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 -- 172.21.15.145:0/1376281740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 msgr2=0x5592e1864de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e1864de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 -- 172.21.15.145:0/1376281740 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e15cdde0 con 0x5592e1854050 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e1857480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:52:06.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5ff659c700 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8011ab0 tx=0x7f5fe8011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc2e52ea99e114a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5fe57fa700 1 -- 172.21.15.145:0/1376281740 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800e9a0 con 0x5592e1854050 2023-08-25T04:52:06.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.777+0000 7f5fe57fa700 1 -- 172.21.15.145:0/1376281740 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe8021070 con 0x5592e1854050 2023-08-25T04:52:06.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f5fe57fa700 1 -- 172.21.15.145:0/1376281740 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80055e0 con 0x5592e1854050 2023-08-25T04:52:06.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 -- 172.21.15.145:0/1376281740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 msgr2=0x5592e185d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8011ab0 tx=0x7f5fe8011b60 comp rx=0 tx=0).stop 2023-08-25T04:52:06.786 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 -- 172.21.15.145:0/1376281740 shutdown_connections 2023-08-25T04:52:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e185a260 0x5592e1857480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e1854050 0x5592e185d470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 --2- 172.21.15.145:0/1376281740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e1864de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 -- 172.21.15.145:0/1376281740 >> 172.21.15.145:0/1376281740 conn(0x5592e17c40e0 msgr2=0x5592e185cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.787 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 -- 172.21.15.145:0/1376281740 shutdown_connections 2023-08-25T04:52:06.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.778+0000 7f601075db80 1 -- 172.21.15.145:0/1376281740 wait complete. 2023-08-25T04:52:06.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.779+0000 7f601075db80 1 -- start start 2023-08-25T04:52:06.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.779+0000 7f601075db80 1 Processor -- start 2023-08-25T04:52:06.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.788 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e18ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e18b6070 con 0x5592e1859940 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e18b61e0 con 0x5592e1854050 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e18b63a0 con 0x5592e185a260 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f601075db80 10 client.0 _ll_register_callbacks cb 0x7f5ff0094500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:52:06.789 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:44882/0 (socket says 172.21.15.145:44882) 2023-08-25T04:52:06.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:46792/0 (socket says 172.21.15.145:46792) 2023-08-25T04:52:06.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/3424461758 learned_addr learned my addr 172.21.15.145:0/3424461758 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:06.790 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff5d9b700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e18ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/3424461758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 msgr2=0x5592e18b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 msgr2=0x5592e18ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:06.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e18ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.791 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff6d9d700 1 -- 172.21.15.145:0/3424461758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8017070 con 0x5592e185a260 2023-08-25T04:52:06.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.780+0000 7f5ff659c700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:52:06.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f5ff6d9d700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0094de0 tx=0x7f5ff0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6827a72966024194 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:06.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f5fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:52:06.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000ce60 con 0x5592e185a260 2023-08-25T04:52:06.792 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9433+0+0 (secure 0 0 0) 0x7f5ff00988e0 con 0x5592e185a260 2023-08-25T04:52:06.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592e18428b0 con 0x5592e185a260 2023-08-25T04:52:06.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff009c710 con 0x5592e185a260 2023-08-25T04:52:06.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f601075db80 10 client.26002 fetch_fsmap learned FSMap version 35 2023-08-25T04:52:06.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.781+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5592e15b6c60 con 0x5592e185a260 2023-08-25T04:52:06.793 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592e18428b0 con 0x5592e185a260 2023-08-25T04:52:06.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ff00a01d0 con 0x5592e185a260 2023-08-25T04:52:06.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 10 client.26002 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:52:06.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:52:06.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 20 client.26002 populate_metadata read hostname 'smithi145' 2023-08-25T04:52:06.794 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e18c29f0 0x5592e18c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:06.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 4 client.26002 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:52:06.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5592e179a970 con 0x5592e18c29f0 2023-08-25T04:52:06.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.782+0000 7f5ff659c700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e18c29f0 0x5592e18c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:06.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f5ff659c700 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e18c29f0 0x5592e18c4e10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f483f9b936f3e912 server_cookie=ce748087cb762393 in_seq=0 out_seq=0 2023-08-25T04:52:06.795 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f5fddffb700 10 client.26002 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:06.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f5fddffb700 1 -- 172.21.15.145:0/3424461758 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5592e179a970 con 0x5592e18c29f0 2023-08-25T04:52:06.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f5fddffb700 10 client.26002 handle_command_reply: tid=0 2023-08-25T04:52:06.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 client.26002 shutdown 2023-08-25T04:52:06.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 msgr2=0x5592e18b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.796 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0094de0 tx=0x7f5ff0004ad0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 shutdown_connections 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5592e18c29f0 0x5592e18c4e10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592e185a260 0x5592e18b5960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5592e1854050 0x5592e18b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 --2- 172.21.15.145:0/3424461758 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5592e1859940 0x5592e18ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.797 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f5fddffb700 0 client.26002 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:06.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 >> 172.21.15.145:0/3424461758 conn(0x5592e17c40e0 msgr2=0x5592e1857480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 shutdown_connections 2023-08-25T04:52:06.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.783+0000 7f601075db80 1 -- 172.21.15.145:0/3424461758 wait complete. 2023-08-25T04:52:06.798 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f601075db80 20 client.26002 trim_cache size 0 max 16384 2023-08-25T04:52:06.799 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:52:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:52:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:52:06.799 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:52:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:52:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:52:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:52:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:52:06.800 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:52:06.801 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:52:06.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 msgr2=0x7f5fe00a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 0x7f5fe00a26a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8009260 comp rx=0 tx=0).stop 2023-08-25T04:52:06.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 msgr2=0x7f5ff819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:06.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000ef30 tx=0x7f5ff000c3a0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 shutdown_connections 2023-08-25T04:52:06.802 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff8106780 0x7f5ff819afd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5fe00a01e0 0x7f5fe00a26a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ff8067a80 0x7f5ff81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 --2- 172.21.15.145:0/1759630129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ff8068440 0x7f5ff8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:06.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.784+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 >> 172.21.15.145:0/1759630129 conn(0x7f5ff80750e0 msgr2=0x7f5ff80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:06.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.785+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 shutdown_connections 2023-08-25T04:52:06.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:06.785+0000 7f5ff759e700 1 -- 172.21.15.145:0/1759630129 wait complete. 2023-08-25T04:52:06.806 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:52:07.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:06 smithi162 ceph-mon[130941]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T04:52:07.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:06 smithi183 ceph-mon[131414]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T04:52:07.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:06 smithi145 ceph-mon[97681]: osdmap e91: 12 total, 12 up, 12 in 2023-08-25T04:52:08.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:07 smithi162 ceph-mon[130941]: pgmap v215: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:07 smithi183 ceph-mon[131414]: pgmap v215: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:07 smithi145 ceph-mon[97681]: pgmap v215: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:09 smithi162 ceph-mon[130941]: pgmap v216: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:09 smithi183 ceph-mon[131414]: pgmap v216: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:09 smithi145 ceph-mon[97681]: pgmap v216: 129 pgs: 129 active+clean; 8.0 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:52:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:11 smithi162 ceph-mon[130941]: pgmap v217: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:11 smithi183 ceph-mon[131414]: pgmap v217: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:11 smithi145 ceph-mon[97681]: pgmap v217: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:13 smithi162 ceph-mon[130941]: pgmap v218: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:13 smithi183 ceph-mon[131414]: pgmap v218: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:14.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:13 smithi145 ceph-mon[97681]: pgmap v218: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:15 smithi145 ceph-mon[97681]: pgmap v219: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:15 smithi145 ceph-mon[97681]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T04:52:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:15 smithi162 ceph-mon[130941]: pgmap v219: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:15 smithi162 ceph-mon[130941]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T04:52:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:15 smithi183 ceph-mon[131414]: pgmap v219: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-08-25T04:52:16.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:15 smithi183 ceph-mon[131414]: osdmap e92: 12 total, 12 up, 12 in 2023-08-25T04:52:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:17 smithi145 ceph-mon[97681]: pgmap v221: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:17 smithi162 ceph-mon[130941]: pgmap v221: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:18.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:17 smithi183 ceph-mon[131414]: pgmap v221: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:19 smithi145 ceph-mon[97681]: pgmap v222: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:19 smithi162 ceph-mon[130941]: pgmap v222: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:19 smithi183 ceph-mon[131414]: pgmap v222: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:52:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:21 smithi145 ceph-mon[97681]: pgmap v223: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:21 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:52:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:21 smithi162 ceph-mon[130941]: pgmap v223: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:21 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:52:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:21 smithi183 ceph-mon[131414]: pgmap v223: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:21 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:52:23.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:22 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:52:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:22 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:52:23.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:22 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:52:23.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:22 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:52:23.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:22 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:52:23.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:22 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:52:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:22 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:52:23.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:22 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:52:23.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:22 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:52:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:23 smithi162 ceph-mon[130941]: pgmap v224: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:23 smithi183 ceph-mon[131414]: pgmap v224: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:23 smithi145 ceph-mon[97681]: pgmap v224: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:25 smithi162 ceph-mon[130941]: pgmap v225: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:25 smithi183 ceph-mon[131414]: pgmap v225: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:25 smithi145 ceph-mon[97681]: pgmap v225: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:52:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:27 smithi162 ceph-mon[130941]: pgmap v226: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:52:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:27 smithi183 ceph-mon[131414]: pgmap v226: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:52:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:27 smithi145 ceph-mon[97681]: pgmap v226: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:52:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:29 smithi162 ceph-mon[130941]: pgmap v227: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:29 smithi183 ceph-mon[131414]: pgmap v227: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:30.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:29 smithi145 ceph-mon[97681]: pgmap v227: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:32.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:31 smithi162 ceph-mon[130941]: pgmap v228: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:31 smithi183 ceph-mon[131414]: pgmap v228: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:31 smithi145 ceph-mon[97681]: pgmap v228: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:33 smithi162 ceph-mon[130941]: pgmap v229: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:33 smithi183 ceph-mon[131414]: pgmap v229: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:33 smithi145 ceph-mon[97681]: pgmap v229: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:35 smithi183 ceph-mon[131414]: pgmap v230: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:35 smithi145 ceph-mon[97681]: pgmap v230: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:35 smithi162 ceph-mon[130941]: pgmap v230: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:36.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:52:37.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 -- 172.21.15.145:0/259599412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 msgr2=0x7fdf1010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 --2- 172.21.15.145:0/259599412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf1010f570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800b0a0 tx=0x7fdf0800b3b0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 -- 172.21.15.145:0/259599412 shutdown_connections 2023-08-25T04:52:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 --2- 172.21.15.145:0/259599412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf1010f570 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 --2- 172.21.15.145:0/259599412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10072cf0 0x7fdf100730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 --2- 172.21.15.145:0/259599412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10073610 0x7fdf1010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 -- 172.21.15.145:0/259599412 >> 172.21.15.145:0/259599412 conn(0x7fdf100fc390 msgr2=0x7fdf100fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 -- 172.21.15.145:0/259599412 shutdown_connections 2023-08-25T04:52:37.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.061+0000 7fdf161ff700 1 -- 172.21.15.145:0/259599412 wait complete. 2023-08-25T04:52:37.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.062+0000 7fdf161ff700 1 Processor -- start 2023-08-25T04:52:37.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.062+0000 7fdf161ff700 1 -- start start 2023-08-25T04:52:37.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 0x7fdf1019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10073610 0x7fdf1019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf10198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101991a0 con 0x7fdf10072cf0 2023-08-25T04:52:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10199310 con 0x7fdf10073610 2023-08-25T04:52:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf161ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101994d0 con 0x7fdf1010d140 2023-08-25T04:52:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 0x7fdf1019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf0ffff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf10198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 0x7fdf1019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44482/0 (socket says 172.21.15.145:44482) 2023-08-25T04:52:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1696185573 learned_addr learned my addr 172.21.15.145:0/1696185573 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.063+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10073610 0x7fdf1019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 -- 172.21.15.145:0/1696185573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10073610 msgr2=0x7fdf1019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10073610 0x7fdf1019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 -- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 msgr2=0x7fdf1019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 0x7fdf1019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 -- 172.21.15.145:0/1696185573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08009d00 con 0x7fdf1010d140 2023-08-25T04:52:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0ffff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf10198b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009fd0 tx=0x7fdf08007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0cff9700 1 -- 172.21.15.145:0/1696185573 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800e040 con 0x7fdf1010d140 2023-08-25T04:52:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0cff9700 1 -- 172.21.15.145:0/1696185573 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0801d470 con 0x7fdf1010d140 2023-08-25T04:52:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf0cff9700 1 -- 172.21.15.145:0/1696185573 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080052a0 con 0x7fdf1010d140 2023-08-25T04:52:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf10199750 con 0x7fdf1010d140 2023-08-25T04:52:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.064+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101a5ab0 con 0x7fdf1010d140 2023-08-25T04:52:37.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.068+0000 7fdf0cff9700 1 -- 172.21.15.145:0/1696185573 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fdf0801d5e0 con 0x7fdf1010d140 2023-08-25T04:52:37.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.069+0000 7fdf29b17b80 1 Processor -- start 2023-08-25T04:52:37.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 -- start start 2023-08-25T04:52:37.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653dd9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 0x563653dd40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653d3b7b0 con 0x563653dd3c40 2023-08-25T04:52:37.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653d2c780 con 0x563653dd9380 2023-08-25T04:52:37.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.070+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653d2a0a0 con 0x563653dd9ca0 2023-08-25T04:52:37.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0cff9700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 0x7fdef80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653dd9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653dd9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54114/0 (socket says 172.21.15.145:54114) 2023-08-25T04:52:37.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0effd700 1 -- 172.21.15.145:0/1134921576 learned_addr learned my addr 172.21.15.145:0/1134921576 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:37.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0cff9700 1 -- 172.21.15.145:0/1696185573 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fdf080ccc40 con 0x7fdf1010d140 2023-08-25T04:52:37.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 0x563653dd40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1134921576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 msgr2=0x563653dd9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653dd9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0ffff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 0x7fdef80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1134921576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 msgr2=0x563653dd40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 0x563653dd40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1134921576 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563653ba5630 con 0x563653dd9ca0 2023-08-25T04:52:37.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 0x563653dd40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:52:37.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.071+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd3700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009d10 tx=0x7fdf00011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d831fd421335e0a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.072+0000 7fdef5ffb700 1 -- 172.21.15.145:0/1134921576 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004250 con 0x563653dd9ca0 2023-08-25T04:52:37.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.072+0000 7fdef5ffb700 1 -- 172.21.15.145:0/1134921576 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00004cf0 con 0x563653dd9ca0 2023-08-25T04:52:37.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.072+0000 7fdef5ffb700 1 -- 172.21.15.145:0/1134921576 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000055e0 con 0x563653dd9ca0 2023-08-25T04:52:37.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1134921576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 msgr2=0x563653dd3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd3700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009d10 tx=0x7fdf00011720 comp rx=0 tx=0).stop 2023-08-25T04:52:37.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf0ffff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 0x7fdef80a2600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf08096ab0 tx=0x7fdf0800b040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1134921576 shutdown_connections 2023-08-25T04:52:37.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd3c40 0x563653dd40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653dd9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1134921576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd3700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.089 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1134921576 >> 172.21.15.145:0/1134921576 conn(0x563653d480e0 msgr2=0x563653dce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1134921576 shutdown_connections 2023-08-25T04:52:37.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1134921576 wait complete. 2023-08-25T04:52:37.090 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.073+0000 7fdf29b17b80 1 -- start start 2023-08-25T04:52:37.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 Processor -- start 2023-08-25T04:52:37.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 0x563653ddfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9ca0 0x563653ddd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653d25990 con 0x563653dd9ca0 2023-08-25T04:52:37.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653cc3bb0 con 0x563653dd9380 2023-08-25T04:52:37.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653cc38b0 con 0x563653dd3c40 2023-08-25T04:52:37.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.074+0000 7fdf29b17b80 10 client.0 _ll_register_callbacks cb 0x563653c0da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:52:37.094 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 0x563653ddfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 0x563653ddfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34408/0 (socket says 172.21.15.145:34408) 2023-08-25T04:52:37.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0effd700 1 -- 172.21.15.145:0/1788336171 learned_addr learned my addr 172.21.15.145:0/1788336171 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:37.095 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54116/0 (socket says 172.21.15.145:54116) 2023-08-25T04:52:37.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9ca0 0x563653ddd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1788336171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 msgr2=0x563653ddfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 0x563653ddfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.096 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9ca0 msgr2=0x563653ddd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9ca0 0x563653ddd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.075+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/1788336171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00009970 con 0x563653dd9380 2023-08-25T04:52:37.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf00000c00 tx=0x7fdf00005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391bef14cd6b98f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.097 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdeeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:52:37.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000053f0 con 0x563653dd9380 2023-08-25T04:52:37.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563653da31c0 con 0x563653dd9380 2023-08-25T04:52:37.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fdf00005550 con 0x563653dd9380 2023-08-25T04:52:37.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00005710 con 0x563653dd9380 2023-08-25T04:52:37.098 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdf29b17b80 10 client.25985 fetch_fsmap learned FSMap version 35 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.076+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563653b51de0 con 0x563653dd9380 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563653da31c0 con 0x563653dd9380 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf000039f0 con 0x563653dd9380 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 10 client.25985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:52:37.099 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 20 client.25985 populate_metadata read hostname 'smithi145' 2023-08-25T04:52:37.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653ded6a0 0x563653defac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 4 client.25985 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:52:37.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.077+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563653ccbba0 con 0x563653ded6a0 2023-08-25T04:52:37.100 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.078+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653ded6a0 0x563653defac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.078+0000 7fdf0effd700 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653ded6a0 0x563653defac0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee08aa753de396e server_cookie=a7ea5544f361ae60 in_seq=0 out_seq=0 2023-08-25T04:52:37.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.078+0000 7fdeeeffd700 10 client.25985 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:37.101 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.078+0000 7fdeeeffd700 1 -- 172.21.15.145:0/1788336171 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563653ccbba0 con 0x563653ded6a0 2023-08-25T04:52:37.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdeeeffd700 10 client.25985 handle_command_reply: tid=0 2023-08-25T04:52:37.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 client.25985 shutdown 2023-08-25T04:52:37.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 msgr2=0x563653ddcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf00000c00 tx=0x7fdf00005ce0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.102 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 shutdown_connections 2023-08-25T04:52:37.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9ca0 0x563653ddd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653ded6a0 0x563653defac0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd3c40 0x563653ddfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/1788336171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd9380 0x563653ddcee0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.103 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 >> 172.21.15.145:0/1788336171 conn(0x563653d480e0 msgr2=0x563653dc8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdeeeffd700 0 client.25985 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:37.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 shutdown_connections 2023-08-25T04:52:37.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 1 -- 172.21.15.145:0/1788336171 wait complete. 2023-08-25T04:52:37.104 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.079+0000 7fdf29b17b80 20 client.25985 trim_cache size 0 max 16384 2023-08-25T04:52:37.106 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 Processor -- start 2023-08-25T04:52:37.107 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 -- start start 2023-08-25T04:52:37.108 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653dd3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.109 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653ddc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653ba7920 con 0x563653dd9380 2023-08-25T04:52:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653ddde90 con 0x563653dd3c40 2023-08-25T04:52:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.106+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653dcaa20 con 0x563653dd9ca0 2023-08-25T04:52:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.110 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653ddc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653ddc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34410/0 (socket says 172.21.15.145:34410) 2023-08-25T04:52:37.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44512/0 (socket says 172.21.15.145:44512) 2023-08-25T04:52:37.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0effd700 1 -- 172.21.15.145:0/2513149418 learned_addr learned my addr 172.21.15.145:0/2513149418 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:37.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0effd700 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653dd3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/2513149418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 msgr2=0x563653ddc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.113 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653ddc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/2513149418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 msgr2=0x563653dd3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653dd3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/2513149418 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563653b3ac60 con 0x563653dd9380 2023-08-25T04:52:37.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.107+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009c60 tx=0x7fdf00011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f670b06166d8af9b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdef67fc700 1 -- 172.21.15.145:0/2513149418 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00005d50 con 0x563653dd9380 2023-08-25T04:52:37.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdef67fc700 1 -- 172.21.15.145:0/2513149418 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00015070 con 0x563653dd9380 2023-08-25T04:52:37.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdef67fc700 1 -- 172.21.15.145:0/2513149418 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00003680 con 0x563653dd9380 2023-08-25T04:52:37.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 -- 172.21.15.145:0/2513149418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 msgr2=0x563653ddd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009c60 tx=0x7fdf00011ba0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 -- 172.21.15.145:0/2513149418 shutdown_connections 2023-08-25T04:52:37.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653ddc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653dd3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/2513149418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653ddd5e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 -- 172.21.15.145:0/2513149418 >> 172.21.15.145:0/2513149418 conn(0x563653d480e0 msgr2=0x563653de9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.108+0000 7fdf29b17b80 1 -- 172.21.15.145:0/2513149418 shutdown_connections 2023-08-25T04:52:37.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.109+0000 7fdf29b17b80 1 -- 172.21.15.145:0/2513149418 wait complete. 2023-08-25T04:52:37.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.109+0000 7fdf29b17b80 1 -- start start 2023-08-25T04:52:37.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.109+0000 7fdf29b17b80 1 Processor -- start 2023-08-25T04:52:37.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653de4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.130 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653dd5c00 con 0x563653dd9380 2023-08-25T04:52:37.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653dd5d70 con 0x563653dd3c40 2023-08-25T04:52:37.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x563653dd5f30 con 0x563653dd9ca0 2023-08-25T04:52:37.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf29b17b80 10 client.0 _ll_register_callbacks cb 0x7fdf04009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:52:37.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:34426/0 (socket says 172.21.15.145:34426) 2023-08-25T04:52:37.133 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44518/0 (socket says 172.21.15.145:44518) 2023-08-25T04:52:37.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0effd700 1 -- 172.21.15.145:0/3690097007 learned_addr learned my addr 172.21.15.145:0/3690097007 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:52:37.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.110+0000 7fdf0effd700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653de4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.134 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/3690097007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 msgr2=0x563653dd5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.135 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/3690097007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 msgr2=0x563653de4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:52:37.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653de4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 -- 172.21.15.145:0/3690097007 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563653b51de0 con 0x563653dd9380 2023-08-25T04:52:37.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0effd700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:52:37.136 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf0f7fe700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x563653dcead0 tx=0x7fdf00009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa15f8d468c6af7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:52:37.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdeeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:52:37.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00005600 con 0x563653dd9380 2023-08-25T04:52:37.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563653dc68b0 con 0x563653dd9380 2023-08-25T04:52:37.137 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fdf00003680 con 0x563653dd9380 2023-08-25T04:52:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00003860 con 0x563653dd9380 2023-08-25T04:52:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.111+0000 7fdf29b17b80 10 client.15993 fetch_fsmap learned FSMap version 35 2023-08-25T04:52:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563653b78250 con 0x563653dd9380 2023-08-25T04:52:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563653dc68b0 con 0x563653dd9380 2023-08-25T04:52:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf00004680 con 0x563653dd9380 2023-08-25T04:52:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:52:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:52:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 20 client.15993 populate_metadata read hostname 'smithi145' 2023-08-25T04:52:37.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653e418f0 0x563653e43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:52:37.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 4 client.15993 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:52:37.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.112+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563653d1e970 con 0x563653e418f0 2023-08-25T04:52:37.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdf0effd700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653e418f0 0x563653e43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:52:37.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdf0effd700 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653e418f0 0x563653e43ce0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a5e2441cbfed40 server_cookie=9cff0a9964280974 in_seq=0 out_seq=0 2023-08-25T04:52:37.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdeeeffd700 10 client.15993 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:37.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdeeeffd700 1 -- 172.21.15.145:0/3690097007 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563653d1e970 con 0x563653e418f0 2023-08-25T04:52:37.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdeeeffd700 10 client.15993 handle_command_reply: tid=0 2023-08-25T04:52:37.143 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.113+0000 7fdf29b17b80 1 client.15993 shutdown 2023-08-25T04:52:37.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 msgr2=0x563653dd4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x563653dcead0 tx=0x7fdf00009810 comp rx=0 tx=0).stop 2023-08-25T04:52:37.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 shutdown_connections 2023-08-25T04:52:37.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x563653e418f0 0x563653e43ce0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563653dd9ca0 0x563653dd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563653dd3c40 0x563653de4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 --2- 172.21.15.145:0/3690097007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563653dd9380 0x563653dd4dc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 >> 172.21.15.145:0/3690097007 conn(0x563653d480e0 msgr2=0x563653de8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdeeeffd700 0 client.15993 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:52:37.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 shutdown_connections 2023-08-25T04:52:37.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 1 -- 172.21.15.145:0/3690097007 wait complete. 2023-08-25T04:52:37.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.114+0000 7fdf29b17b80 20 client.15993 trim_cache size 0 max 16384 2023-08-25T04:52:37.149 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:52:37.149 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:52:37.149 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:52:37.150 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:52:37.150 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:52:37.150 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:52:37.150 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:52:37.151 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:52:37.151 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:52:37.151 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:52:37.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 msgr2=0x7fdef80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 0x7fdef80a2600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf08096ab0 tx=0x7fdf0800b040 comp rx=0 tx=0).stop 2023-08-25T04:52:37.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 msgr2=0x7fdf10198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:52:37.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf10198b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009fd0 tx=0x7fdf08007930 comp rx=0 tx=0).stop 2023-08-25T04:52:37.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 shutdown_connections 2023-08-25T04:52:37.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1010d140 0x7fdf10198b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf10072cf0 0x7fdf1019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fdef80a0140 0x7fdef80a2600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 --2- 172.21.15.145:0/1696185573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf10073610 0x7fdf1019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:52:37.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 >> 172.21.15.145:0/1696185573 conn(0x7fdf100fc390 msgr2=0x7fdf10104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:52:37.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 shutdown_connections 2023-08-25T04:52:37.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:52:37.115+0000 7fdf161ff700 1 -- 172.21.15.145:0/1696185573 wait complete. 2023-08-25T04:52:37.156 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:52:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:37 smithi183 ceph-mon[131414]: pgmap v231: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:38.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:37 smithi145 ceph-mon[97681]: pgmap v231: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:37 smithi162 ceph-mon[130941]: pgmap v231: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:40.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:39 smithi183 ceph-mon[131414]: pgmap v232: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:39 smithi145 ceph-mon[97681]: pgmap v232: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:40.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:39 smithi162 ceph-mon[130941]: pgmap v232: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:41 smithi183 ceph-mon[131414]: pgmap v233: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:42.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:41 smithi145 ceph-mon[97681]: pgmap v233: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:41 smithi162 ceph-mon[130941]: pgmap v233: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:43 smithi145 ceph-mon[97681]: pgmap v234: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:44.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:43 smithi162 ceph-mon[130941]: pgmap v234: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:44.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:43 smithi183 ceph-mon[131414]: pgmap v234: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:46.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:45 smithi145 ceph-mon[97681]: pgmap v235: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:46.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:45 smithi162 ceph-mon[130941]: pgmap v235: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:46.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:45 smithi183 ceph-mon[131414]: pgmap v235: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:48.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:47 smithi145 ceph-mon[97681]: pgmap v236: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:48.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:47 smithi162 ceph-mon[130941]: pgmap v236: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:47 smithi183 ceph-mon[131414]: pgmap v236: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:50.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:49 smithi145 ceph-mon[97681]: pgmap v237: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:50.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:49 smithi162 ceph-mon[130941]: pgmap v237: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:50.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:49 smithi183 ceph-mon[131414]: pgmap v237: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:52.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:51 smithi145 ceph-mon[97681]: pgmap v238: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:52.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:51 smithi162 ceph-mon[130941]: pgmap v238: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:52.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:51 smithi183 ceph-mon[131414]: pgmap v238: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:54.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:53 smithi145 ceph-mon[97681]: pgmap v239: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:53 smithi162 ceph-mon[130941]: pgmap v239: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:54.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:53 smithi183 ceph-mon[131414]: pgmap v239: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:56.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:55 smithi145 ceph-mon[97681]: pgmap v240: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:56.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:55 smithi162 ceph-mon[130941]: pgmap v240: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:56.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:55 smithi183 ceph-mon[131414]: pgmap v240: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:52:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:57 smithi145 ceph-mon[97681]: pgmap v241: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:57 smithi162 ceph-mon[130941]: pgmap v241: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:52:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:57 smithi183 ceph-mon[131414]: pgmap v241: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:00.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:52:59 smithi145 ceph-mon[97681]: pgmap v242: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:52:59 smithi162 ceph-mon[130941]: pgmap v242: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:52:59 smithi183 ceph-mon[131414]: pgmap v242: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:01 smithi145 ceph-mon[97681]: pgmap v243: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:02.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:01 smithi162 ceph-mon[130941]: pgmap v243: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:01 smithi183 ceph-mon[131414]: pgmap v243: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:03 smithi145 ceph-mon[97681]: pgmap v244: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:04.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:03 smithi162 ceph-mon[130941]: pgmap v244: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:03 smithi183 ceph-mon[131414]: pgmap v244: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:05 smithi145 ceph-mon[97681]: pgmap v245: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:06.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:05 smithi162 ceph-mon[130941]: pgmap v245: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:05 smithi183 ceph-mon[131414]: pgmap v245: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:07.157 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:53:07.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:06 smithi145 ceph-mon[97681]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T04:53:07.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/4266729494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8101a30 msgr2=0x7fc8a8101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/4266729494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8101a30 0x7fc8a8101e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc898009990 tx=0x7fc898009ca0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/4266729494 shutdown_connections 2023-08-25T04:53:07.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/4266729494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 0x7fc8a810d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/4266729494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8101a30 0x7fc8a8101e10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/4266729494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8102350 0x7fc8a810a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.387+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/4266729494 >> 172.21.15.145:0/4266729494 conn(0x7fc8a80fb270 msgr2=0x7fc8a80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.388+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/4266729494 shutdown_connections 2023-08-25T04:53:07.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.388+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/4266729494 wait complete. 2023-08-25T04:53:07.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 Processor -- start 2023-08-25T04:53:07.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 -- start start 2023-08-25T04:53:07.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.392 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8102350 0x7fc8a819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 0x7fc8a8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8199460 con 0x7fc8a8102350 2023-08-25T04:53:07.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81995d0 con 0x7fc8a8101a30 2023-08-25T04:53:07.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.389+0000 7fc8ae9ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8199790 con 0x7fc8a810ae50 2023-08-25T04:53:07.393 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:57706/0 (socket says 172.21.15.145:57706) 2023-08-25T04:53:07.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 -- 172.21.15.145:0/2170031802 learned_addr learned my addr 172.21.15.145:0/2170031802 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:07.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8102350 0x7fc8a819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 0x7fc8a8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.394 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 -- 172.21.15.145:0/2170031802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 msgr2=0x7fc8a8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 0x7fc8a8198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 -- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8102350 msgr2=0x7fc8a819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8102350 0x7fc8a819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 -- 172.21.15.145:0/2170031802 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980095f0 con 0x7fc8a8101a30 2023-08-25T04:53:07.395 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.390+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc898005910 tx=0x7fc898004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.391+0000 7fc8a57fa700 1 -- 172.21.15.145:0/2170031802 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89801d070 con 0x7fc8a8101a30 2023-08-25T04:53:07.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.391+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a8199a10 con 0x7fc8a8101a30 2023-08-25T04:53:07.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.391+0000 7fc8a57fa700 1 -- 172.21.15.145:0/2170031802 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc898022470 con 0x7fc8a8101a30 2023-08-25T04:53:07.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.391+0000 7fc8a57fa700 1 -- 172.21.15.145:0/2170031802 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898017650 con 0x7fc8a8101a30 2023-08-25T04:53:07.396 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.391+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81a59d0 con 0x7fc8a8101a30 2023-08-25T04:53:07.398 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.394+0000 7fc8a57fa700 1 -- 172.21.15.145:0/2170031802 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fc8980225e0 con 0x7fc8a8101a30 2023-08-25T04:53:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.395+0000 7fc8c232eb80 1 Processor -- start 2023-08-25T04:53:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 -- start start 2023-08-25T04:53:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 0x5603784fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784f5c40 0x5603784f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037845d7b0 con 0x5603784fbca0 2023-08-25T04:53:07.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037844e780 con 0x5603784f5c40 2023-08-25T04:53:07.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037844c0a0 con 0x5603784fb380 2023-08-25T04:53:07.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8a57fa700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 0x7fc8900a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.400 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8a57fa700 1 -- 172.21.15.145:0/2170031802 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fc8980ccc60 con 0x7fc8a8101a30 2023-08-25T04:53:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.396+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 0x5603784fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 0x5603784fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60650/0 (socket says 172.21.15.145:60650) 2023-08-25T04:53:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8acf9c700 1 -- 172.21.15.145:0/3149754454 learned_addr learned my addr 172.21.15.145:0/3149754454 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 0x7fc8900a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784f5c40 0x5603784f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 -- 172.21.15.145:0/3149754454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 msgr2=0x5603784fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 0x5603784fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 -- 172.21.15.145:0/3149754454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784f5c40 msgr2=0x5603784f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784f5c40 0x5603784f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 -- 172.21.15.145:0/3149754454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603782c7630 con 0x5603784fbca0 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 0x7fc8900a2580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009fd0 tx=0x7fc8a000b040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.397+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f5700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8980c9fc0 tx=0x7fc898022cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb72755fa29011 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc88e7fc700 1 -- 172.21.15.145:0/3149754454 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898099970 con 0x5603784fbca0 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc88e7fc700 1 -- 172.21.15.145:0/3149754454 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89809a710 con 0x5603784fbca0 2023-08-25T04:53:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc88e7fc700 1 -- 172.21.15.145:0/3149754454 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89809a4d0 con 0x5603784fbca0 2023-08-25T04:53:07.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 -- 172.21.15.145:0/3149754454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 msgr2=0x5603784f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f5700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8980c9fc0 tx=0x7fc898022cd0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 -- 172.21.15.145:0/3149754454 shutdown_connections 2023-08-25T04:53:07.407 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784f5c40 0x5603784f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.408 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fb380 0x5603784fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.408 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/3149754454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f5700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.408 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.398+0000 7fc8c232eb80 1 -- 172.21.15.145:0/3149754454 >> 172.21.15.145:0/3149754454 conn(0x56037846a0e0 msgr2=0x5603784f0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.408 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.399+0000 7fc8c232eb80 1 -- 172.21.15.145:0/3149754454 shutdown_connections 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.399+0000 7fc8c232eb80 1 -- 172.21.15.145:0/3149754454 wait complete. 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.399+0000 7fc8c232eb80 1 -- start start 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.399+0000 7fc8c232eb80 1 Processor -- start 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378501c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.409 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560378447990 con 0x5603784f5c40 2023-08-25T04:53:07.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603783e5bb0 con 0x5603784fb380 2023-08-25T04:53:07.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603783e58b0 con 0x5603784fbca0 2023-08-25T04:53:07.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8c232eb80 10 client.0 _ll_register_callbacks cb 0x56037832fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:53:07.410 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378501c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60666/0 (socket says 172.21.15.145:60666) 2023-08-25T04:53:07.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8acf9c700 1 -- 172.21.15.145:0/1060551439 learned_addr learned my addr 172.21.15.145:0/1060551439 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:07.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.400+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.411 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 -- 172.21.15.145:0/1060551439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 msgr2=0x5603784ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 -- 172.21.15.145:0/1060551439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 msgr2=0x5603784feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 -- 172.21.15.145:0/1060551439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81a59d0 con 0x5603784f5c40 2023-08-25T04:53:07.412 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:53:07.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378501c30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc8980c9fc0 tx=0x7fc898022b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14be0b893f1287a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8877fe700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:53:07.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89809a010 con 0x5603784f5c40 2023-08-25T04:53:07.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fc89809de40 con 0x5603784f5c40 2023-08-25T04:53:07.413 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.401+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603784c51c0 con 0x5603784f5c40 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8980a1d90 con 0x5603784f5c40 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 10 client.16005 fetch_fsmap learned FSMap version 35 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560378273de0 con 0x5603784f5c40 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8980b1070 con 0x5603784f5c40 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc89800fb10 con 0x5603784f5c40 2023-08-25T04:53:07.414 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:53:07.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:53:07.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 20 client.16005 populate_metadata read hostname 'smithi145' 2023-08-25T04:53:07.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56037850ee80 0x560378511270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 4 client.16005 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:53:07.415 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.402+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603783edba0 con 0x56037850ee80 2023-08-25T04:53:07.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.403+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56037850ee80 0x560378511270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.403+0000 7fc8877fe700 10 client.16005 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:07.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.403+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56037850ee80 0x560378511270 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e830402dd004e8 server_cookie=5c2d0baab1ff4594 in_seq=0 out_seq=0 2023-08-25T04:53:07.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8877fe700 1 -- 172.21.15.145:0/1060551439 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603783edba0 con 0x56037850ee80 2023-08-25T04:53:07.416 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8877fe700 10 client.16005 handle_command_reply: tid=0 2023-08-25T04:53:07.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 client.16005 shutdown 2023-08-25T04:53:07.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 msgr2=0x560378501c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378501c30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc8980c9fc0 tx=0x7fc898022b30 comp rx=0 tx=0).stop 2023-08-25T04:53:07.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 shutdown_connections 2023-08-25T04:53:07.417 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x56037850ee80 0x560378511270 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378501c30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/1060551439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 >> 172.21.15.145:0/1060551439 conn(0x56037846a0e0 msgr2=0x5603784eab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.418 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8877fe700 0 client.16005 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:07.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.404+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 shutdown_connections 2023-08-25T04:53:07.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.405+0000 7fc8c232eb80 1 -- 172.21.15.145:0/1060551439 wait complete. 2023-08-25T04:53:07.419 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.405+0000 7fc8c232eb80 20 client.16005 trim_cache size 0 max 16384 2023-08-25T04:53:07.420 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:06 smithi162 ceph-mon[130941]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T04:53:07.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:06 smithi183 ceph-mon[131414]: osdmap e93: 12 total, 12 up, 12 in 2023-08-25T04:53:07.427 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 Processor -- start 2023-08-25T04:53:07.427 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 -- start start 2023-08-25T04:53:07.427 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.427 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603782c9920 con 0x5603784fbca0 2023-08-25T04:53:07.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603784ec810 con 0x5603784fb380 2023-08-25T04:53:07.428 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.422+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603784ecc80 con 0x5603784f5c40 2023-08-25T04:53:07.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60682/0 (socket says 172.21.15.145:60682) 2023-08-25T04:53:07.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 -- 172.21.15.145:0/411300288 learned_addr learned my addr 172.21.15.145:0/411300288 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:07.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 -- 172.21.15.145:0/411300288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 msgr2=0x5603784f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 -- 172.21.15.145:0/411300288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 msgr2=0x5603784f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 -- 172.21.15.145:0/411300288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56037825cc60 con 0x5603784f5c40 2023-08-25T04:53:07.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc898095d90 tx=0x7fc898022cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f46a186af29359 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc88effd700 1 -- 172.21.15.145:0/411300288 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980a1d90 con 0x5603784f5c40 2023-08-25T04:53:07.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.423+0000 7fc88effd700 1 -- 172.21.15.145:0/411300288 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8980b1020 con 0x5603784f5c40 2023-08-25T04:53:07.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc88effd700 1 -- 172.21.15.145:0/411300288 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800fb10 con 0x5603784f5c40 2023-08-25T04:53:07.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 -- 172.21.15.145:0/411300288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 msgr2=0x5603785017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc898095d90 tx=0x7fc898022cd0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 -- 172.21.15.145:0/411300288 shutdown_connections 2023-08-25T04:53:07.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784fbca0 0x5603784f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784f5c40 0x5603785017d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/411300288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 -- 172.21.15.145:0/411300288 >> 172.21.15.145:0/411300288 conn(0x56037846a0e0 msgr2=0x560378510e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 -- 172.21.15.145:0/411300288 shutdown_connections 2023-08-25T04:53:07.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.424+0000 7fc8c232eb80 1 -- 172.21.15.145:0/411300288 wait complete. 2023-08-25T04:53:07.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.425+0000 7fc8c232eb80 1 -- start start 2023-08-25T04:53:07.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.425+0000 7fc8c232eb80 1 Processor -- start 2023-08-25T04:53:07.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378506620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.434 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603784f7980 con 0x5603784f5c40 2023-08-25T04:53:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603784f7af0 con 0x5603784fb380 2023-08-25T04:53:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603784f7cb0 con 0x5603784fbca0 2023-08-25T04:53:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8c232eb80 10 client.0 _ll_register_callbacks cb 0x7fc89c009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:53:07.435 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378506620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:60698/0 (socket says 172.21.15.145:60698) 2023-08-25T04:53:07.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 -- 172.21.15.145:0/165563983 learned_addr learned my addr 172.21.15.145:0/165563983 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:07.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378506620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54588/0 (socket says 172.21.15.145:54588) 2023-08-25T04:53:07.436 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8a77fe700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 -- 172.21.15.145:0/165563983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 msgr2=0x5603784f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 -- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 msgr2=0x560378506620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:07.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378506620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.426+0000 7fc8acf9c700 1 -- 172.21.15.145:0/165563983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560378273de0 con 0x5603784fbca0 2023-08-25T04:53:07.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8acf9c700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8a001b700 tx=0x7fc8a0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9da4a790d60d2b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:07.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8877fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:53:07.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0021040 con 0x5603784fbca0 2023-08-25T04:53:07.441 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603784e88b0 con 0x5603784fbca0 2023-08-25T04:53:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fc8a0004bb0 con 0x5603784fbca0 2023-08-25T04:53:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a000e070 con 0x5603784fbca0 2023-08-25T04:53:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8c232eb80 10 client.26038 fetch_fsmap learned FSMap version 35 2023-08-25T04:53:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56037829a250 con 0x5603784fbca0 2023-08-25T04:53:07.442 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.427+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603784e88b0 con 0x5603784fbca0 2023-08-25T04:53:07.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8a002e020 con 0x5603784fbca0 2023-08-25T04:53:07.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 10 client.26038 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:53:07.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 10 client.26038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:53:07.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 20 client.26038 populate_metadata read hostname 'smithi145' 2023-08-25T04:53:07.443 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5603785633d0 0x5603785657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:07.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 4 client.26038 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:53:07.444 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560378440970 con 0x5603785633d0 2023-08-25T04:53:07.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5603785633d0 0x5603785657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:07.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8a7fff700 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5603785633d0 0x5603785657f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebe8743a24db681 server_cookie=a4582f41a1aee333 in_seq=0 out_seq=0 2023-08-25T04:53:07.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.428+0000 7fc8877fe700 10 client.26038 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:07.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8877fe700 1 -- 172.21.15.145:0/165563983 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560378440970 con 0x5603785633d0 2023-08-25T04:53:07.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8877fe700 10 client.26038 handle_command_reply: tid=0 2023-08-25T04:53:07.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 client.26038 shutdown 2023-08-25T04:53:07.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 msgr2=0x5603784f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8a001b700 tx=0x7fc8a0004ad0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 shutdown_connections 2023-08-25T04:53:07.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5603785633d0 0x5603785657f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603784fbca0 0x5603784f72e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603784f5c40 0x560378506620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 --2- 172.21.15.145:0/165563983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603784fb380 0x5603784f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 >> 172.21.15.145:0/165563983 conn(0x56037846a0e0 msgr2=0x5603784f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8877fe700 0 client.26038 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:07.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 shutdown_connections 2023-08-25T04:53:07.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 1 -- 172.21.15.145:0/165563983 wait complete. 2023-08-25T04:53:07.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.429+0000 7fc8c232eb80 20 client.26038 trim_cache size 0 max 16384 2023-08-25T04:53:07.451 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:53:07.451 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:53:07.452 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:53:07.452 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:53:07.452 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:53:07.453 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:53:07.453 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:53:07.453 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:53:07.454 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:53:07.454 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:53:07.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 msgr2=0x7fc8900a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.456 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 0x7fc8900a2580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009fd0 tx=0x7fc8a000b040 comp rx=0 tx=0).stop 2023-08-25T04:53:07.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 msgr2=0x7fc8a819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:07.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc898005910 tx=0x7fc898004710 comp rx=0 tx=0).stop 2023-08-25T04:53:07.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 shutdown_connections 2023-08-25T04:53:07.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a810ae50 0x7fc8a8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8900a00c0 0x7fc8900a2580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a8101a30 0x7fc8a819e9e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 --2- 172.21.15.145:0/2170031802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8a8102350 0x7fc8a819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:07.458 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 >> 172.21.15.145:0/2170031802 conn(0x7fc8a80fb270 msgr2=0x7fc8a80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:07.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 shutdown_connections 2023-08-25T04:53:07.459 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:07.430+0000 7fc8ae9ff700 1 -- 172.21.15.145:0/2170031802 wait complete. 2023-08-25T04:53:07.459 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:53:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:07 smithi145 ceph-mon[97681]: pgmap v247: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:08.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:07 smithi162 ceph-mon[130941]: pgmap v247: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:07 smithi183 ceph-mon[131414]: pgmap v247: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:09 smithi145 ceph-mon[97681]: pgmap v248: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:10.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:09 smithi162 ceph-mon[130941]: pgmap v248: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:10.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:09 smithi183 ceph-mon[131414]: pgmap v248: 129 pgs: 129 active+clean; 7.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:11 smithi145 ceph-mon[97681]: pgmap v249: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:11 smithi162 ceph-mon[130941]: pgmap v249: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:11 smithi183 ceph-mon[131414]: pgmap v249: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:13 smithi145 ceph-mon[97681]: pgmap v250: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:14.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:13 smithi162 ceph-mon[130941]: pgmap v250: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:13 smithi183 ceph-mon[131414]: pgmap v250: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:15 smithi145 ceph-mon[97681]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:15 smithi145 ceph-mon[97681]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T04:53:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:15 smithi162 ceph-mon[130941]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:15 smithi162 ceph-mon[130941]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T04:53:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:15 smithi183 ceph-mon[131414]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:53:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:15 smithi183 ceph-mon[131414]: osdmap e94: 12 total, 12 up, 12 in 2023-08-25T04:53:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:17 smithi145 ceph-mon[97681]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:17 smithi162 ceph-mon[130941]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:17 smithi183 ceph-mon[131414]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:20.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:19 smithi145 ceph-mon[97681]: pgmap v254: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:19 smithi162 ceph-mon[130941]: pgmap v254: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:19 smithi183 ceph-mon[131414]: pgmap v254: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:53:22.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:21 smithi145 ceph-mon[97681]: pgmap v255: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:21 smithi162 ceph-mon[130941]: pgmap v255: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:21 smithi183 ceph-mon[131414]: pgmap v255: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:23.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:22 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:53:23.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:22 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:53:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:22 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:53:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:23 smithi145 ceph-mon[97681]: pgmap v256: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:23 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:53:24.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:23 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:53:24.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:23 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:53:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:23 smithi162 ceph-mon[130941]: pgmap v256: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:23 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:53:24.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:23 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:53:24.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:23 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:53:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:23 smithi183 ceph-mon[131414]: pgmap v256: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:24.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:23 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:53:24.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:23 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:53:24.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:23 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:53:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:25 smithi145 ceph-mon[97681]: pgmap v257: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:25 smithi162 ceph-mon[130941]: pgmap v257: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:25 smithi183 ceph-mon[131414]: pgmap v257: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:53:28.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:27 smithi145 ceph-mon[97681]: pgmap v258: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:53:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:27 smithi162 ceph-mon[130941]: pgmap v258: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:53:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:27 smithi183 ceph-mon[131414]: pgmap v258: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T04:53:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:29 smithi145 ceph-mon[97681]: pgmap v259: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:29 smithi162 ceph-mon[130941]: pgmap v259: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:29 smithi183 ceph-mon[131414]: pgmap v259: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:31 smithi145 ceph-mon[97681]: pgmap v260: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:31 smithi162 ceph-mon[130941]: pgmap v260: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:31 smithi183 ceph-mon[131414]: pgmap v260: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:34.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:33 smithi145 ceph-mon[97681]: pgmap v261: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:33 smithi162 ceph-mon[130941]: pgmap v261: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:33 smithi183 ceph-mon[131414]: pgmap v261: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:35 smithi145 ceph-mon[97681]: pgmap v262: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:36.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:35 smithi162 ceph-mon[130941]: pgmap v262: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:35 smithi183 ceph-mon[131414]: pgmap v262: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:37.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:53:37.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 -- 172.21.15.145:0/1405014165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9141010d0 msgr2=0x7ff91410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1405014165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9141010d0 0x7ff91410d5e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00b0a0 tx=0x7ff90c00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 -- 172.21.15.145:0/1405014165 shutdown_connections 2023-08-25T04:53:37.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1405014165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9141010d0 0x7ff91410d5e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1405014165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914106720 0x7ff914106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1405014165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914100710 0x7ff914100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 -- 172.21.15.145:0/1405014165 >> 172.21.15.145:0/1405014165 conn(0x7ff9140fc1e0 msgr2=0x7ff9140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 -- 172.21.15.145:0/1405014165 shutdown_connections 2023-08-25T04:53:37.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.720+0000 7ff919b9d700 1 -- 172.21.15.145:0/1405014165 wait complete. 2023-08-25T04:53:37.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.721+0000 7ff919b9d700 1 Processor -- start 2023-08-25T04:53:37.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 -- start start 2023-08-25T04:53:37.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914100710 0x7ff91419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 0x7ff91419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 0x7ff9141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914198d60 con 0x7ff9141010d0 2023-08-25T04:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914198ed0 con 0x7ff914100710 2023-08-25T04:53:37.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff919b9d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914199090 con 0x7ff914106720 2023-08-25T04:53:37.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff913fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 0x7ff91419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff91939c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 0x7ff9141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff913fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 0x7ff91419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:39468/0 (socket says 172.21.15.145:39468) 2023-08-25T04:53:37.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff913fff700 1 -- 172.21.15.145:0/1072304613 learned_addr learned my addr 172.21.15.145:0/1072304613 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:37.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.722+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914100710 0x7ff91419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 -- 172.21.15.145:0/1072304613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914100710 msgr2=0x7ff91419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914100710 0x7ff91419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 -- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 msgr2=0x7ff91419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 0x7ff91419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 -- 172.21.15.145:0/1072304613 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009d00 con 0x7ff914106720 2023-08-25T04:53:37.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff91939c700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 0x7ff9141986b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009fd0 tx=0x7ff90c0075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff911ffb700 1 -- 172.21.15.145:0/1072304613 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00e040 con 0x7ff914106720 2023-08-25T04:53:37.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff911ffb700 1 -- 172.21.15.145:0/1072304613 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90c004c50 con 0x7ff914106720 2023-08-25T04:53:37.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff914199310 con 0x7ff914106720 2023-08-25T04:53:37.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff911ffb700 1 -- 172.21.15.145:0/1072304613 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c01dea0 con 0x7ff914106720 2023-08-25T04:53:37.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.723+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141a5650 con 0x7ff914106720 2023-08-25T04:53:37.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.727+0000 7ff911ffb700 1 -- 172.21.15.145:0/1072304613 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7ff90c01d470 con 0x7ff914106720 2023-08-25T04:53:37.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.728+0000 7ff911ffb700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 0x7ff9040a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.728+0000 7ff911ffb700 1 -- 172.21.15.145:0/1072304613 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7ff90c0ccf80 con 0x7ff914106720 2023-08-25T04:53:37.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.728+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 0x7ff9040a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.729+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 0x7ff9040a26a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff908007b60 tx=0x7ff9080073d0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 Processor -- start 2023-08-25T04:53:37.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 -- start start 2023-08-25T04:53:37.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f0c40 0x55beda1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda1587b0 con 0x55beda1f6e20 2023-08-25T04:53:37.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda149780 con 0x55beda1f0c40 2023-08-25T04:53:37.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff92e685b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda1470a0 con 0x55beda1f6500 2023-08-25T04:53:37.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55910/0 (socket says 172.21.15.145:55910) 2023-08-25T04:53:37.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.730+0000 7ff918b9b700 1 -- 172.21.15.145:0/1437213896 learned_addr learned my addr 172.21.15.145:0/1437213896 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:37.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff91939c700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f0c40 0x55beda1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff913fff700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 -- 172.21.15.145:0/1437213896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f0c40 msgr2=0x55beda1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f0c40 0x55beda1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 -- 172.21.15.145:0/1437213896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 msgr2=0x55beda1f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 -- 172.21.15.145:0/1437213896 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bed9fc2630 con 0x55beda1f6500 2023-08-25T04:53:37.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff913fff700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:53:37.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff918b9b700 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff908010d90 tx=0x7ff908013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f2e85e5b1f87ca8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.731+0000 7ff8feffd700 1 -- 172.21.15.145:0/1437213896 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908011bb0 con 0x55beda1f6500 2023-08-25T04:53:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff8feffd700 1 -- 172.21.15.145:0/1437213896 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff908015070 con 0x55beda1f6500 2023-08-25T04:53:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff8feffd700 1 -- 172.21.15.145:0/1437213896 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90801c610 con 0x55beda1f6500 2023-08-25T04:53:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 -- 172.21.15.145:0/1437213896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 msgr2=0x55beda1f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff908010d90 tx=0x7ff908013000 comp rx=0 tx=0).stop 2023-08-25T04:53:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 -- 172.21.15.145:0/1437213896 shutdown_connections 2023-08-25T04:53:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f0c40 0x55beda1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6500 0x55beda1f68e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 --2- 172.21.15.145:0/1437213896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 -- 172.21.15.145:0/1437213896 >> 172.21.15.145:0/1437213896 conn(0x55beda1650e0 msgr2=0x55beda1eb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.732+0000 7ff92e685b80 1 -- 172.21.15.145:0/1437213896 shutdown_connections 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.733+0000 7ff92e685b80 1 -- 172.21.15.145:0/1437213896 wait complete. 2023-08-25T04:53:37.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.733+0000 7ff92e685b80 1 -- start start 2023-08-25T04:53:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.734+0000 7ff92e685b80 1 Processor -- start 2023-08-25T04:53:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda142990 con 0x55beda1f6e20 2023-08-25T04:53:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda0e0bb0 con 0x55beda1f6500 2023-08-25T04:53:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda0e08b0 con 0x55beda1f0c40 2023-08-25T04:53:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff92e685b80 10 client.0 _ll_register_callbacks cb 0x55beda02aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:53:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55912/0 (socket says 172.21.15.145:55912) 2023-08-25T04:53:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff918b9b700 1 -- 172.21.15.145:0/2994034061 learned_addr learned my addr 172.21.15.145:0/2994034061 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff91939c700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.735+0000 7ff913fff700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 -- 172.21.15.145:0/2994034061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 msgr2=0x55beda1fa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1fa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 -- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 msgr2=0x55beda1fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 -- 172.21.15.145:0/2994034061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908010a40 con 0x55beda1f0c40 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff91939c700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9080109e0 tx=0x7ff90801ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1232aabd3005ae4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff8f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:53:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90802b070 con 0x55beda1f0c40 2023-08-25T04:53:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7ff908017070 con 0x55beda1f0c40 2023-08-25T04:53:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beda1c01c0 con 0x55beda1f0c40 2023-08-25T04:53:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff908011dc0 con 0x55beda1f0c40 2023-08-25T04:53:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff92e685b80 10 client.26050 fetch_fsmap learned FSMap version 35 2023-08-25T04:53:37.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.736+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bed9f6ede0 con 0x55beda1f0c40 2023-08-25T04:53:37.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beda1c01c0 con 0x55beda1f0c40 2023-08-25T04:53:37.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff90801e420 con 0x55beda1f0c40 2023-08-25T04:53:37.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:53:37.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:53:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 20 client.26050 populate_metadata read hostname 'smithi145' 2023-08-25T04:53:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda20a7b0 0x55beda20cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 4 client.26050 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:53:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.737+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55beda0e8ba0 con 0x55beda20a7b0 2023-08-25T04:53:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.738+0000 7ff913fff700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda20a7b0 0x55beda20cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.738+0000 7ff913fff700 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda20a7b0 0x55beda20cbd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7476b93a6b3e98 server_cookie=a55f3b5a08e8ecd6 in_seq=0 out_seq=0 2023-08-25T04:53:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.738+0000 7ff8f3fff700 10 client.26050 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.739+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2994034061 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55beda0e8ba0 con 0x55beda20a7b0 2023-08-25T04:53:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.739+0000 7ff8f3fff700 10 client.26050 handle_command_reply: tid=0 2023-08-25T04:53:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.739+0000 7ff92e685b80 1 client.26050 shutdown 2023-08-25T04:53:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.739+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 msgr2=0x55beda1fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.739+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9080109e0 tx=0x7ff90801ff70 comp rx=0 tx=0).stop 2023-08-25T04:53:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 shutdown_connections 2023-08-25T04:53:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6e20 0x55beda1fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda20a7b0 0x55beda20cbd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda1fce40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2994034061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff8f3fff700 0 client.26050 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 >> 172.21.15.145:0/2994034061 conn(0x55beda1650e0 msgr2=0x55beda1ef6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 shutdown_connections 2023-08-25T04:53:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 1 -- 172.21.15.145:0/2994034061 wait complete. 2023-08-25T04:53:37.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.740+0000 7ff92e685b80 20 client.26050 trim_cache size 0 max 16384 2023-08-25T04:53:37.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.757+0000 7ff92e685b80 1 Processor -- start 2023-08-25T04:53:37.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.757+0000 7ff92e685b80 1 -- start start 2023-08-25T04:53:37.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 0x55beda1f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 0x55beda2019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed9fc4920 con 0x55beda1f0c40 2023-08-25T04:53:37.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda1fb550 con 0x55beda1f6500 2023-08-25T04:53:37.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff92e685b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda1f1990 con 0x55beda1f6e20 2023-08-25T04:53:37.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 0x55beda1f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff913fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff91939c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 0x55beda2019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 0x55beda1f9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:39482/0 (socket says 172.21.15.145:39482) 2023-08-25T04:53:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff918b9b700 1 -- 172.21.15.145:0/2272022240 learned_addr learned my addr 172.21.15.145:0/2272022240 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff91939c700 1 -- 172.21.15.145:0/2272022240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 msgr2=0x55beda1eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff91939c700 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff91939c700 1 -- 172.21.15.145:0/2272022240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 msgr2=0x55beda1f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.758+0000 7ff91939c700 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 0x55beda1f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.759+0000 7ff91939c700 1 -- 172.21.15.145:0/2272022240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beda1e4df0 con 0x55beda1f6e20 2023-08-25T04:53:37.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.759+0000 7ff91939c700 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 0x55beda2019a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0ca330 tx=0x7ff90c096040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3eb36175ed2978 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.760+0000 7ff8ff7fe700 1 -- 172.21.15.145:0/2272022240 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c004dc0 con 0x55beda1f6e20 2023-08-25T04:53:37.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.760+0000 7ff8ff7fe700 1 -- 172.21.15.145:0/2272022240 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90c09a380 con 0x55beda1f6e20 2023-08-25T04:53:37.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.760+0000 7ff8ff7fe700 1 -- 172.21.15.145:0/2272022240 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c09a5e0 con 0x55beda1f6e20 2023-08-25T04:53:37.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 -- 172.21.15.145:0/2272022240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 msgr2=0x55beda2019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 0x55beda2019a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0ca330 tx=0x7ff90c096040 comp rx=0 tx=0).stop 2023-08-25T04:53:37.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 -- 172.21.15.145:0/2272022240 shutdown_connections 2023-08-25T04:53:37.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f6e20 0x55beda2019a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f0c40 0x55beda1f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2272022240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6500 0x55beda1eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 -- 172.21.15.145:0/2272022240 >> 172.21.15.145:0/2272022240 conn(0x55beda1650e0 msgr2=0x55beda1eb280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 -- 172.21.15.145:0/2272022240 shutdown_connections 2023-08-25T04:53:37.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.761+0000 7ff92e685b80 1 -- 172.21.15.145:0/2272022240 wait complete. 2023-08-25T04:53:37.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.762+0000 7ff92e685b80 1 -- start start 2023-08-25T04:53:37.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.762+0000 7ff92e685b80 1 Processor -- start 2023-08-25T04:53:37.769 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6500 0x55beda1ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6e20 0x55beda1fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda2004a0 con 0x55beda1f6500 2023-08-25T04:53:37.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda200610 con 0x55beda1f6e20 2023-08-25T04:53:37.770 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beda2007d0 con 0x55beda1f0c40 2023-08-25T04:53:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff92e685b80 10 client.0 _ll_register_callbacks cb 0x7ff90c0966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:53:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff918b9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55936/0 (socket says 172.21.15.145:55936) 2023-08-25T04:53:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff918b9b700 1 -- 172.21.15.145:0/2315575873 learned_addr learned my addr 172.21.15.145:0/2315575873 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:53:37.771 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff913fff700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6500 0x55beda1ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.763+0000 7ff91939c700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6e20 0x55beda1fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 -- 172.21.15.145:0/2315575873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6e20 msgr2=0x55beda1fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6e20 0x55beda1fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 -- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6500 msgr2=0x55beda1ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:53:37.772 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6500 0x55beda1ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 -- 172.21.15.145:0/2315575873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141a5650 con 0x55beda1f0c40 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff918b9b700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff90801e970 tx=0x7ff90801ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a96118c7a83d2f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff8f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90802b040 con 0x55beda1f0c40 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beda207260 con 0x55beda1f0c40 2023-08-25T04:53:37.773 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7ff908017070 con 0x55beda1f0c40 2023-08-25T04:53:37.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.764+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff908005520 con 0x55beda1f0c40 2023-08-25T04:53:37.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 10 client.26056 fetch_fsmap learned FSMap version 35 2023-08-25T04:53:37.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bed9f57c60 con 0x55beda1f0c40 2023-08-25T04:53:37.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beda207260 con 0x55beda1f0c40 2023-08-25T04:53:37.774 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9080304b0 con 0x55beda1f0c40 2023-08-25T04:53:37.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 10 client.26056 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:53:37.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:53:37.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 20 client.26056 populate_metadata read hostname 'smithi145' 2023-08-25T04:53:37.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda25f540 0x55beda261960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:53:37.775 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 4 client.26056 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:53:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55beda1e38b0 con 0x55beda25f540 2023-08-25T04:53:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.765+0000 7ff913fff700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda25f540 0x55beda261960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:53:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff913fff700 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda25f540 0x55beda261960 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ce3f7186161042 server_cookie=56c334a9f86196bc in_seq=0 out_seq=0 2023-08-25T04:53:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff8f3fff700 10 client.26056 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:37.776 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff8f3fff700 1 -- 172.21.15.145:0/2315575873 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55beda1e38b0 con 0x55beda25f540 2023-08-25T04:53:37.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff8f3fff700 10 client.26056 handle_command_reply: tid=0 2023-08-25T04:53:37.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff92e685b80 1 client.26056 shutdown 2023-08-25T04:53:37.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 msgr2=0x55beda2016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.766+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff90801e970 tx=0x7ff90801ede0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.777 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 shutdown_connections 2023-08-25T04:53:37.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55beda1f6e20 0x55beda1fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55beda25f540 0x55beda261960 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55beda1f0c40 0x55beda2016c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 --2- 172.21.15.145:0/2315575873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55beda1f6500 0x55beda1ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.778 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff8f3fff700 0 client.26056 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:53:37.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 >> 172.21.15.145:0/2315575873 conn(0x55beda1650e0 msgr2=0x55beda1f3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 shutdown_connections 2023-08-25T04:53:37.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 1 -- 172.21.15.145:0/2315575873 wait complete. 2023-08-25T04:53:37.779 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.767+0000 7ff92e685b80 20 client.26056 trim_cache size 0 max 16384 2023-08-25T04:53:37.780 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:53:37.780 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:53:37.780 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:53:37.780 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:53:37.781 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:53:37.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 msgr2=0x7ff9040a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.782 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 0x7ff9040a26a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff908007b60 tx=0x7ff9080073d0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 msgr2=0x7ff9141986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:53:37.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 0x7ff9141986b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009fd0 tx=0x7ff90c0075c0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 shutdown_connections 2023-08-25T04:53:37.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff914106720 0x7ff9141986b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.783 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7ff9040a01e0 0x7ff9040a26a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff914100710 0x7ff91419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 --2- 172.21.15.145:0/1072304613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9141010d0 0x7ff91419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:53:37.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 >> 172.21.15.145:0/1072304613 conn(0x7ff9140fc1e0 msgr2=0x7ff914072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:53:37.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 shutdown_connections 2023-08-25T04:53:37.784 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:53:37.768+0000 7ff919b9d700 1 -- 172.21.15.145:0/1072304613 wait complete. 2023-08-25T04:53:37.786 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:37 smithi145 ceph-mon[97681]: pgmap v263: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:37.789 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:53:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:37 smithi162 ceph-mon[130941]: pgmap v263: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:37 smithi183 ceph-mon[131414]: pgmap v263: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:40.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:39 smithi145 ceph-mon[97681]: pgmap v264: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:39 smithi162 ceph-mon[130941]: pgmap v264: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:39 smithi183 ceph-mon[131414]: pgmap v264: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:42.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:41 smithi145 ceph-mon[97681]: pgmap v265: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:42.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:41 smithi162 ceph-mon[130941]: pgmap v265: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:41 smithi183 ceph-mon[131414]: pgmap v265: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:44.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:43 smithi145 ceph-mon[97681]: pgmap v266: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:43 smithi162 ceph-mon[130941]: pgmap v266: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:43 smithi183 ceph-mon[131414]: pgmap v266: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:46.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:45 smithi145 ceph-mon[97681]: pgmap v267: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:45 smithi162 ceph-mon[130941]: pgmap v267: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:45 smithi183 ceph-mon[131414]: pgmap v267: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:48.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:47 smithi145 ceph-mon[97681]: pgmap v268: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:48.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:47 smithi162 ceph-mon[130941]: pgmap v268: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:47 smithi183 ceph-mon[131414]: pgmap v268: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:50.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:49 smithi145 ceph-mon[97681]: pgmap v269: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:50.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:49 smithi162 ceph-mon[130941]: pgmap v269: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:49 smithi183 ceph-mon[131414]: pgmap v269: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:51 smithi145 ceph-mon[97681]: pgmap v270: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:51 smithi162 ceph-mon[130941]: pgmap v270: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:51 smithi183 ceph-mon[131414]: pgmap v270: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:54.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:53 smithi145 ceph-mon[97681]: pgmap v271: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:53 smithi162 ceph-mon[130941]: pgmap v271: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:53 smithi183 ceph-mon[131414]: pgmap v271: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:55 smithi145 ceph-mon[97681]: pgmap v272: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:55 smithi162 ceph-mon[130941]: pgmap v272: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:55 smithi183 ceph-mon[131414]: pgmap v272: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:53:58.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:57 smithi145 ceph-mon[97681]: pgmap v273: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:57 smithi162 ceph-mon[130941]: pgmap v273: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:53:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:57 smithi183 ceph-mon[131414]: pgmap v273: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:00.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:53:59 smithi145 ceph-mon[97681]: pgmap v274: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:53:59 smithi162 ceph-mon[130941]: pgmap v274: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:53:59 smithi183 ceph-mon[131414]: pgmap v274: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:02.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:01 smithi145 ceph-mon[97681]: pgmap v275: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:01 smithi162 ceph-mon[130941]: pgmap v275: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:01 smithi183 ceph-mon[131414]: pgmap v275: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:04.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:03 smithi145 ceph-mon[97681]: pgmap v276: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:03 smithi162 ceph-mon[130941]: pgmap v276: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:03 smithi183 ceph-mon[131414]: pgmap v276: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:05 smithi145 ceph-mon[97681]: pgmap v277: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:05 smithi162 ceph-mon[130941]: pgmap v277: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:05 smithi183 ceph-mon[131414]: pgmap v277: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:07.790 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:54:08.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 -- 172.21.15.145:0/2490522099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1410ae50 msgr2=0x7f2f1410d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/2490522099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1410ae50 0x7f2f1410d310 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b0a0 tx=0x7f2f0c00b3b0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 -- 172.21.15.145:0/2490522099 shutdown_connections 2023-08-25T04:54:08.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/2490522099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f1410ae50 0x7f2f1410d310 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/2490522099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14101a30 0x7f2f14101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/2490522099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f14102350 0x7f2f1410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 -- 172.21.15.145:0/2490522099 >> 172.21.15.145:0/2490522099 conn(0x7f2f140fb270 msgr2=0x7f2f140fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 -- 172.21.15.145:0/2490522099 shutdown_connections 2023-08-25T04:54:08.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.032+0000 7f2f1bbff700 1 -- 172.21.15.145:0/2490522099 wait complete. 2023-08-25T04:54:08.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.033+0000 7f2f1bbff700 1 Processor -- start 2023-08-25T04:54:08.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 -- start start 2023-08-25T04:54:08.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 0x7f2f140ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1410ae50 0x7f2f141a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141003f0 con 0x7f2f14102350 2023-08-25T04:54:08.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141a51f0 con 0x7f2f1410ae50 2023-08-25T04:54:08.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1bbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141a5330 con 0x7f2f14101a30 2023-08-25T04:54:08.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49760/0 (socket says 172.21.15.145:49760) 2023-08-25T04:54:08.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.034+0000 7f2f1999b700 1 -- 172.21.15.145:0/1631031417 learned_addr learned my addr 172.21.15.145:0/1631031417 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:08.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1410ae50 0x7f2f141a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1919a700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 0x7f2f140ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 -- 172.21.15.145:0/1631031417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1410ae50 msgr2=0x7f2f141a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1410ae50 0x7f2f141a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 -- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 msgr2=0x7f2f140ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 0x7f2f140ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 -- 172.21.15.145:0/1631031417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009d00 con 0x7f2f14101a30 2023-08-25T04:54:08.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1919a700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 0x7f2f140ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:54:08.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1999b700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e9a0 tx=0x7f2f1000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f0affd700 1 -- 172.21.15.145:0/1631031417 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10009790 con 0x7f2f14101a30 2023-08-25T04:54:08.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f141a5590 con 0x7f2f14101a30 2023-08-25T04:54:08.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f0affd700 1 -- 172.21.15.145:0/1631031417 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1000cbd0 con 0x7f2f14101a30 2023-08-25T04:54:08.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f0affd700 1 -- 172.21.15.145:0/1631031417 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10018680 con 0x7f2f14101a30 2023-08-25T04:54:08.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.035+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141a5a70 con 0x7f2f14101a30 2023-08-25T04:54:08.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.039+0000 7f2f0affd700 1 -- 172.21.15.145:0/1631031417 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f2f1000cd40 con 0x7f2f14101a30 2023-08-25T04:54:08.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.040+0000 7f2f2f4b5b80 1 Processor -- start 2023-08-25T04:54:08.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.040+0000 7f2f2f4b5b80 1 -- start start 2023-08-25T04:54:08.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f0affd700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 0x7f2f000a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 0x558938b52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 0x558938b4d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938ab47b0 con 0x558938b4cc40 2023-08-25T04:54:08.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938aa5780 con 0x558938b52ca0 2023-08-25T04:54:08.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938aa30a0 con 0x558938b52380 2023-08-25T04:54:08.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f0affd700 1 -- 172.21.15.145:0/1631031417 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f2f10014070 con 0x7f2f14101a30 2023-08-25T04:54:08.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 0x558938b52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1919a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1a19c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 0x558938b4d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1919a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:33102/0 (socket says 172.21.15.145:33102) 2023-08-25T04:54:08.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1919a700 1 -- 172.21.15.145:0/2711543574 learned_addr learned my addr 172.21.15.145:0/2711543574 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:08.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.041+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 0x7f2f000a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 -- 172.21.15.145:0/2711543574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 msgr2=0x558938b52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 0x558938b52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 -- 172.21.15.145:0/2711543574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 msgr2=0x558938b4d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 0x558938b4d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 -- 172.21.15.145:0/2711543574 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55893891e630 con 0x558938b52ca0 2023-08-25T04:54:08.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1999b700 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 0x558938b52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:54:08.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 0x558938b4d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:54:08.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009d50 tx=0x7f2f040106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2985a96be655eb1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2efffff700 1 -- 172.21.15.145:0/2711543574 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04009ea0 con 0x558938b52ca0 2023-08-25T04:54:08.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2efffff700 1 -- 172.21.15.145:0/2711543574 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f04015070 con 0x558938b52ca0 2023-08-25T04:54:08.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.042+0000 7f2efffff700 1 -- 172.21.15.145:0/2711543574 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0401a5e0 con 0x558938b52ca0 2023-08-25T04:54:08.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2711543574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 msgr2=0x558938b4c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009d50 tx=0x7f2f040106b0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 0x7f2f000a2580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f14101790 tx=0x7f2f0c006230 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2711543574 shutdown_connections 2023-08-25T04:54:08.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b4cc40 0x558938b4d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52380 0x558938b52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2711543574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52ca0 0x558938b4c700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2711543574 >> 172.21.15.145:0/2711543574 conn(0x558938ac10e0 msgr2=0x558938b47410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2711543574 shutdown_connections 2023-08-25T04:54:08.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.043+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2711543574 wait complete. 2023-08-25T04:54:08.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 -- start start 2023-08-25T04:54:08.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 Processor -- start 2023-08-25T04:54:08.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938a9e990 con 0x558938b52ca0 2023-08-25T04:54:08.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938a3cbb0 con 0x558938b52380 2023-08-25T04:54:08.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938a3c8b0 con 0x558938b4cc40 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f2f4b5b80 10 client.0 _ll_register_callbacks cb 0x558938986a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.044+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49776/0 (socket says 172.21.15.145:49776) 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 -- 172.21.15.145:0/2791014523 learned_addr learned my addr 172.21.15.145:0/2791014523 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 -- 172.21.15.145:0/2791014523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 msgr2=0x558938b55f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 -- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 msgr2=0x558938b565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 -- 172.21.15.145:0/2791014523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141a5a70 con 0x558938b4cc40 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f1999b700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f10000c00 tx=0x7f2f10098fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96f75cb0381a8ab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2efd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100a0df0 con 0x558938b4cc40 2023-08-25T04:54:08.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f2f100a0960 con 0x558938b4cc40 2023-08-25T04:54:08.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.045+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558938b1c1c0 con 0x558938b4cc40 2023-08-25T04:54:08.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f100a0b20 con 0x558938b4cc40 2023-08-25T04:54:08.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 10 client.26068 fetch_fsmap learned FSMap version 35 2023-08-25T04:54:08.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f2f04009980 con 0x558938b4cc40 2023-08-25T04:54:08.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558938b1c1c0 con 0x558938b4cc40 2023-08-25T04:54:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f100b8100 con 0x558938b4cc40 2023-08-25T04:54:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:54:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:54:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 20 client.26068 populate_metadata read hostname 'smithi145' 2023-08-25T04:54:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938b66710 0x558938b68b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 4 client.26068 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:54:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558938a44ba0 con 0x558938b66710 2023-08-25T04:54:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.046+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938b66710 0x558938b68b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.047+0000 7f2f1919a700 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938b66710 0x558938b68b00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b058a56ed64a612a server_cookie=8f5cc40669225bfa in_seq=0 out_seq=0 2023-08-25T04:54:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.047+0000 7f2efd7fa700 10 client.26068 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2efd7fa700 1 -- 172.21.15.145:0/2791014523 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558938a44ba0 con 0x558938b66710 2023-08-25T04:54:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2efd7fa700 10 client.26068 handle_command_reply: tid=0 2023-08-25T04:54:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 client.26068 shutdown 2023-08-25T04:54:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 msgr2=0x558938b58c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f10000c00 tx=0x7f2f10098fe0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 shutdown_connections 2023-08-25T04:54:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938b66710 0x558938b68b00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b58c30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/2791014523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2efd7fa700 0 client.26068 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 >> 172.21.15.145:0/2791014523 conn(0x558938ac10e0 msgr2=0x558938b41b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 shutdown_connections 2023-08-25T04:54:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.048+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/2791014523 wait complete. 2023-08-25T04:54:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.049+0000 7f2f2f4b5b80 20 client.26068 trim_cache size 0 max 16384 2023-08-25T04:54:08.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.066+0000 7f2f2f4b5b80 1 Processor -- start 2023-08-25T04:54:08.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.066+0000 7f2f2f4b5b80 1 -- start start 2023-08-25T04:54:08.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.066+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.066+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b55d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938920920 con 0x558938b52ca0 2023-08-25T04:54:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938b43c80 con 0x558938b52380 2023-08-25T04:54:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938b63e10 con 0x558938b4cc40 2023-08-25T04:54:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49792/0 (socket says 172.21.15.145:49792) 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 -- 172.21.15.145:0/3416159946 learned_addr learned my addr 172.21.15.145:0/3416159946 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b55d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1919a700 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 -- 172.21.15.145:0/3416159946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 msgr2=0x558938b55810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 -- 172.21.15.145:0/3416159946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 msgr2=0x558938b55d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b55d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 -- 172.21.15.145:0/3416159946 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558938b6d720 con 0x558938b4cc40 2023-08-25T04:54:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f1999b700 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f10095a50 tx=0x7f2f1009ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9559c44d1b6889ff server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.067+0000 7f2f08bff700 1 -- 172.21.15.145:0/3416159946 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100a0df0 con 0x558938b4cc40 2023-08-25T04:54:08.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f08bff700 1 -- 172.21.15.145:0/3416159946 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f100b8070 con 0x558938b4cc40 2023-08-25T04:54:08.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f08bff700 1 -- 172.21.15.145:0/3416159946 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10097f40 con 0x558938b4cc40 2023-08-25T04:54:08.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/3416159946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 msgr2=0x558938b587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f10095a50 tx=0x7f2f1009ced0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.075 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/3416159946 shutdown_connections 2023-08-25T04:54:08.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52ca0 0x558938b55d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b4cc40 0x558938b587d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/3416159946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b52380 0x558938b55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.076 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/3416159946 >> 172.21.15.145:0/3416159946 conn(0x558938ac10e0 msgr2=0x558938b50050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/3416159946 shutdown_connections 2023-08-25T04:54:08.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.068+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/3416159946 wait complete. 2023-08-25T04:54:08.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 -- start start 2023-08-25T04:54:08.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 Processor -- start 2023-08-25T04:54:08.077 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 0x558938b4f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 0x558938b4de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938b4e340 con 0x558938b52380 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938b4e480 con 0x558938b4cc40 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558938b4e640 con 0x558938b52ca0 2023-08-25T04:54:08.078 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.069+0000 7f2f2f4b5b80 10 client.0 _ll_register_callbacks cb 0x7f2f10098940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:54:08.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1919a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 0x558938b4f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:33134/0 (socket says 172.21.15.145:33134) 2023-08-25T04:54:08.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 -- 172.21.15.145:0/996967979 learned_addr learned my addr 172.21.15.145:0/996967979 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:08.079 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 0x558938b4de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 -- 172.21.15.145:0/996967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 msgr2=0x558938b4de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 0x558938b4de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 -- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 msgr2=0x558938b4f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:08.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 0x558938b4f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.080 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 -- 172.21.15.145:0/996967979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f04009980 con 0x558938b4cc40 2023-08-25T04:54:08.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1919a700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 0x558938b4f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:54:08.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1a19c700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 0x558938b4de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:54:08.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2f1999b700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f10098ce0 tx=0x7f2f1009cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27e48e4ffce4bf58 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:08.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.070+0000 7f2efd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:54:08.081 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10094fd0 con 0x558938b4cc40 2023-08-25T04:54:08.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558938b3f8b0 con 0x558938b4cc40 2023-08-25T04:54:08.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f2f10095130 con 0x558938b4cc40 2023-08-25T04:54:08.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f10095320 con 0x558938b4cc40 2023-08-25T04:54:08.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 10 client.26045 fetch_fsmap learned FSMap version 35 2023-08-25T04:54:08.082 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5589388cade0 con 0x558938b4cc40 2023-08-25T04:54:08.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558938b3f8b0 con 0x558938b4cc40 2023-08-25T04:54:08.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f10010430 con 0x558938b4cc40 2023-08-25T04:54:08.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:54:08.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:54:08.083 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 20 client.26045 populate_metadata read hostname 'smithi145' 2023-08-25T04:54:08.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938bbaf90 0x558938bbd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:08.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 4 client.26045 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:54:08.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.071+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558938a97970 con 0x558938bbaf90 2023-08-25T04:54:08.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.072+0000 7f2f1919a700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938bbaf90 0x558938bbd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:08.084 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.072+0000 7f2efd7fa700 10 client.26045 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:08.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.072+0000 7f2f1919a700 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938bbaf90 0x558938bbd380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d74059ae2684fab server_cookie=68d22fb75b2e23a9 in_seq=0 out_seq=0 2023-08-25T04:54:08.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.072+0000 7f2efd7fa700 1 -- 172.21.15.145:0/996967979 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558938a97970 con 0x558938bbaf90 2023-08-25T04:54:08.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.072+0000 7f2efd7fa700 10 client.26045 handle_command_reply: tid=0 2023-08-25T04:54:08.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 client.26045 shutdown 2023-08-25T04:54:08.085 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 msgr2=0x558938b4f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f10098ce0 tx=0x7f2f1009cf40 comp rx=0 tx=0).stop 2023-08-25T04:54:08.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 shutdown_connections 2023-08-25T04:54:08.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x558938bbaf90 0x558938bbd380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558938b52ca0 0x558938b4de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.086 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558938b4cc40 0x558938b4f1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 --2- 172.21.15.145:0/996967979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558938b52380 0x558938b4f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 >> 172.21.15.145:0/996967979 conn(0x558938ac10e0 msgr2=0x558938b50fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2efd7fa700 0 client.26045 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:08.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 shutdown_connections 2023-08-25T04:54:08.087 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 1 -- 172.21.15.145:0/996967979 wait complete. 2023-08-25T04:54:08.088 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.073+0000 7f2f2f4b5b80 20 client.26045 trim_cache size 0 max 16384 2023-08-25T04:54:08.088 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:54:08.088 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:54:08.088 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:54:08.089 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:54:08.089 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:54:08.089 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:54:08.089 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:54:08.089 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:54:08.090 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:54:08.090 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:54:08.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 msgr2=0x7f2f000a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 0x7f2f000a2580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f14101790 tx=0x7f2f0c006230 comp rx=0 tx=0).stop 2023-08-25T04:54:08.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 msgr2=0x7f2f140ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:08.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e9a0 tx=0x7f2f1000ecb0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.091 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 shutdown_connections 2023-08-25T04:54:08.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f2f000a00c0 0x7f2f000a2580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1410ae50 0x7f2f141a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f14101a30 0x7f2f140ff680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.092 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 --2- 172.21.15.145:0/1631031417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14102350 0x7f2f140ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:08.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 >> 172.21.15.145:0/1631031417 conn(0x7f2f140fb270 msgr2=0x7f2f1410d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:08.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 shutdown_connections 2023-08-25T04:54:08.093 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:08.074+0000 7f2f1bbff700 1 -- 172.21.15.145:0/1631031417 wait complete. 2023-08-25T04:54:08.093 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:07 smithi145 ceph-mon[97681]: pgmap v278: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:08.094 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:54:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:07 smithi162 ceph-mon[130941]: pgmap v278: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:07 smithi183 ceph-mon[131414]: pgmap v278: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:10.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:09 smithi145 ceph-mon[97681]: pgmap v279: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:54:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:09 smithi162 ceph-mon[130941]: pgmap v279: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:54:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:09 smithi183 ceph-mon[131414]: pgmap v279: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:54:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:11 smithi145 ceph-mon[97681]: pgmap v280: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:11 smithi145 ceph-mon[97681]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T04:54:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:11 smithi162 ceph-mon[130941]: pgmap v280: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:12.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:11 smithi162 ceph-mon[130941]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T04:54:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:11 smithi183 ceph-mon[131414]: pgmap v280: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:54:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:11 smithi183 ceph-mon[131414]: osdmap e95: 12 total, 12 up, 12 in 2023-08-25T04:54:14.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:13 smithi145 ceph-mon[97681]: pgmap v282: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:13 smithi162 ceph-mon[130941]: pgmap v282: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:13 smithi183 ceph-mon[131414]: pgmap v282: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:16.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:15 smithi145 ceph-mon[97681]: pgmap v283: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:15 smithi162 ceph-mon[130941]: pgmap v283: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:15 smithi183 ceph-mon[131414]: pgmap v283: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:54:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:17 smithi145 ceph-mon[97681]: pgmap v284: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:17 smithi162 ceph-mon[130941]: pgmap v284: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:17 smithi183 ceph-mon[131414]: pgmap v284: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:19 smithi145 ceph-mon[97681]: pgmap v285: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:20.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:19 smithi145 ceph-mon[97681]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T04:54:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:19 smithi162 ceph-mon[130941]: pgmap v285: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:19 smithi162 ceph-mon[130941]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T04:54:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:19 smithi183 ceph-mon[131414]: pgmap v285: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:20.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:19 smithi183 ceph-mon[131414]: osdmap e96: 12 total, 12 up, 12 in 2023-08-25T04:54:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:21 smithi162 ceph-mon[130941]: pgmap v287: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 313 B/s wr, 6 op/s 2023-08-25T04:54:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:21 smithi183 ceph-mon[131414]: pgmap v287: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 313 B/s wr, 6 op/s 2023-08-25T04:54:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:21 smithi145 ceph-mon[97681]: pgmap v287: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 313 B/s wr, 6 op/s 2023-08-25T04:54:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:23 smithi162 ceph-mon[130941]: pgmap v288: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:23 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:54:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:23 smithi183 ceph-mon[131414]: pgmap v288: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:24.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:23 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:54:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:23 smithi145 ceph-mon[97681]: pgmap v288: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:23 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:54:25.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:24 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:54:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:24 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:54:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:24 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:54:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:24 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:54:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:24 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:54:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:24 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:54:25.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:24 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:54:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:24 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:54:25.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:24 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:54:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:25 smithi162 ceph-mon[130941]: pgmap v289: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:25 smithi183 ceph-mon[131414]: pgmap v289: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:25 smithi145 ceph-mon[97681]: pgmap v289: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-08-25T04:54:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:27 smithi162 ceph-mon[130941]: pgmap v290: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:27 smithi183 ceph-mon[131414]: pgmap v290: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:27 smithi145 ceph-mon[97681]: pgmap v290: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:29 smithi162 ceph-mon[130941]: pgmap v291: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:29 smithi183 ceph-mon[131414]: pgmap v291: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:29 smithi145 ceph-mon[97681]: pgmap v291: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:54:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:31 smithi162 ceph-mon[130941]: pgmap v292: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 188 B/s wr, 8 op/s 2023-08-25T04:54:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:31 smithi183 ceph-mon[131414]: pgmap v292: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 188 B/s wr, 8 op/s 2023-08-25T04:54:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:31 smithi145 ceph-mon[97681]: pgmap v292: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 188 B/s wr, 8 op/s 2023-08-25T04:54:34.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:33 smithi162 ceph-mon[130941]: pgmap v293: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:33 smithi183 ceph-mon[131414]: pgmap v293: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:33 smithi145 ceph-mon[97681]: pgmap v293: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:35 smithi162 ceph-mon[130941]: pgmap v294: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:35 smithi183 ceph-mon[131414]: pgmap v294: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:35 smithi145 ceph-mon[97681]: pgmap v294: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:38.095 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:54:38.157 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:37 smithi145 ceph-mon[97681]: pgmap v295: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:37 smithi162 ceph-mon[130941]: pgmap v295: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:37 smithi183 ceph-mon[131414]: pgmap v295: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:38.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 -- 172.21.15.145:0/4121267109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 msgr2=0x7f3924106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 --2- 172.21.15.145:0/4121267109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924106c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3914009990 tx=0x7f3914009ca0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 -- 172.21.15.145:0/4121267109 shutdown_connections 2023-08-25T04:54:38.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 --2- 172.21.15.145:0/4121267109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924101230 0x7f392410d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 --2- 172.21.15.145:0/4121267109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924106c60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 --2- 172.21.15.145:0/4121267109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924100870 0x7f3924100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 -- 172.21.15.145:0/4121267109 >> 172.21.15.145:0/4121267109 conn(0x7f39240fc3a0 msgr2=0x7f39240fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 -- 172.21.15.145:0/4121267109 shutdown_connections 2023-08-25T04:54:38.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.310+0000 7f392a3ff700 1 -- 172.21.15.145:0/4121267109 wait complete. 2023-08-25T04:54:38.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.311+0000 7f392a3ff700 1 Processor -- start 2023-08-25T04:54:38.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.311+0000 7f392a3ff700 1 -- start start 2023-08-25T04:54:38.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924100870 0x7f392419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 0x7f392419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241990d0 con 0x7f3924101230 2023-08-25T04:54:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924199240 con 0x7f3924100870 2023-08-25T04:54:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392a3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924199400 con 0x7f3924106880 2023-08-25T04:54:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392899c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f39237fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 0x7f392419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392899c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35864/0 (socket says 172.21.15.145:35864) 2023-08-25T04:54:38.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f3923fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924100870 0x7f392419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.312+0000 7f392899c700 1 -- 172.21.15.145:0/3560641637 learned_addr learned my addr 172.21.15.145:0/3560641637 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:38.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 -- 172.21.15.145:0/3560641637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 msgr2=0x7f3924198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 -- 172.21.15.145:0/3560641637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924100870 msgr2=0x7f392419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924100870 0x7f392419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 -- 172.21.15.145:0/3560641637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140095f0 con 0x7f3924101230 2023-08-25T04:54:38.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39237fe700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 0x7f392419ef30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f391800e9f0 tx=0x7f391800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39217fa700 1 -- 172.21.15.145:0/3560641637 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800c9c0 con 0x7f3924101230 2023-08-25T04:54:38.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39217fa700 1 -- 172.21.15.145:0/3560641637 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3918010430 con 0x7f3924101230 2023-08-25T04:54:38.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39241996e0 con 0x7f3924101230 2023-08-25T04:54:38.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f39217fa700 1 -- 172.21.15.145:0/3560641637 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918018680 con 0x7f3924101230 2023-08-25T04:54:38.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.313+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39241a59e0 con 0x7f3924101230 2023-08-25T04:54:38.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.317+0000 7f39217fa700 1 -- 172.21.15.145:0/3560641637 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f3918010860 con 0x7f3924101230 2023-08-25T04:54:38.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.318+0000 7f39217fa700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 0x7f390c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 Processor -- start 2023-08-25T04:54:38.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f39217fa700 1 -- 172.21.15.145:0/3560641637 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f3918014070 con 0x7f3924101230 2023-08-25T04:54:38.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 -- start start 2023-08-25T04:54:38.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f3923fff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 0x7f390c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374992760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937498d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593748f47b0 con 0x55937498cc40 2023-08-25T04:54:38.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593748e5780 con 0x559374992380 2023-08-25T04:54:38.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.319+0000 7f393de22b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593748e30a0 con 0x559374992ca0 2023-08-25T04:54:38.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35868/0 (socket says 172.21.15.145:35868) 2023-08-25T04:54:38.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 -- 172.21.15.145:0/2955505927 learned_addr learned my addr 172.21.15.145:0/2955505927 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:38.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f392899c700 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937498d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f3923fff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 0x7f390c0a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f391400b2e0 tx=0x7f3914005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 -- 172.21.15.145:0/2955505927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 msgr2=0x559374992760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374992760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 -- 172.21.15.145:0/2955505927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 msgr2=0x55937498d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937498d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 -- 172.21.15.145:0/2955505927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55937475e630 con 0x559374992ca0 2023-08-25T04:54:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f39237fe700 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f39180a0760 tx=0x7f39180988b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59262a824cfe81f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.320+0000 7f390a7fc700 1 -- 172.21.15.145:0/2955505927 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391809ccb0 con 0x559374992ca0 2023-08-25T04:54:38.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f390a7fc700 1 -- 172.21.15.145:0/2955505927 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3918099e90 con 0x559374992ca0 2023-08-25T04:54:38.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f390a7fc700 1 -- 172.21.15.145:0/2955505927 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918098370 con 0x559374992ca0 2023-08-25T04:54:38.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 -- 172.21.15.145:0/2955505927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 msgr2=0x55937498c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f39180a0760 tx=0x7f39180988b0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 -- 172.21.15.145:0/2955505927 shutdown_connections 2023-08-25T04:54:38.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937498d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374992760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 --2- 172.21.15.145:0/2955505927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992ca0 0x55937498c700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 -- 172.21.15.145:0/2955505927 >> 172.21.15.145:0/2955505927 conn(0x5593749010e0 msgr2=0x559374987410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 -- 172.21.15.145:0/2955505927 shutdown_connections 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.321+0000 7f393de22b80 1 -- 172.21.15.145:0/2955505927 wait complete. 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.322+0000 7f393de22b80 1 -- start start 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.322+0000 7f393de22b80 1 Processor -- start 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374995f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559374992ca0 0x559374996660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593748de990 con 0x559374992ca0 2023-08-25T04:54:38.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937487cbb0 con 0x559374992380 2023-08-25T04:54:38.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937487c8b0 con 0x55937498cc40 2023-08-25T04:54:38.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f393de22b80 10 client.0 _ll_register_callbacks cb 0x5593747c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:54:38.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f3923fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f3923fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35880/0 (socket says 172.21.15.145:35880) 2023-08-25T04:54:38.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f3923fff700 1 -- 172.21.15.145:0/2047302219 learned_addr learned my addr 172.21.15.145:0/2047302219 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:38.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f39237fe700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374995f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.323+0000 7f392899c700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559374992ca0 0x559374996660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 -- 172.21.15.145:0/2047302219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 msgr2=0x559374995f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374995f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 -- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559374992ca0 msgr2=0x559374996660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559374992ca0 0x559374996660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 -- 172.21.15.145:0/2047302219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39241a59e0 con 0x55937498cc40 2023-08-25T04:54:38.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f3923fff700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3914005770 tx=0x7f3914009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aab703c8fff1ca6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f39037fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:54:38.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914004df0 con 0x55937498cc40 2023-08-25T04:54:38.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55937495c1c0 con 0x55937498cc40 2023-08-25T04:54:38.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.324+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f391400f460 con 0x55937498cc40 2023-08-25T04:54:38.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39140366a0 con 0x55937498cc40 2023-08-25T04:54:38.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 10 client.26086 fetch_fsmap learned FSMap version 35 2023-08-25T04:54:38.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55937470ade0 con 0x55937498cc40 2023-08-25T04:54:38.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55937495c1c0 con 0x55937498cc40 2023-08-25T04:54:38.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f391400f6b0 con 0x55937498cc40 2023-08-25T04:54:38.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:54:38.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:54:38.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 20 client.26086 populate_metadata read hostname 'smithi145' 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749a67b0 0x5593749a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 4 client.26086 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559374884ba0 con 0x5593749a67b0 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.325+0000 7f39237fe700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749a67b0 0x5593749a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.326+0000 7f39037fe700 10 client.26086 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:38.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.326+0000 7f39237fe700 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749a67b0 0x5593749a8bd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad35dea0a2813ac1 server_cookie=4f09a412c7c07e2a in_seq=0 out_seq=0 2023-08-25T04:54:38.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.326+0000 7f39037fe700 1 -- 172.21.15.145:0/2047302219 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559374884ba0 con 0x5593749a67b0 2023-08-25T04:54:38.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.326+0000 7f39037fe700 10 client.26086 handle_command_reply: tid=0 2023-08-25T04:54:38.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 client.26086 shutdown 2023-08-25T04:54:38.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 msgr2=0x559374998ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3914005770 tx=0x7f3914009310 comp rx=0 tx=0).stop 2023-08-25T04:54:38.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 shutdown_connections 2023-08-25T04:54:38.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559374992ca0 0x559374996660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749a67b0 0x5593749a8bd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55937498cc40 0x559374998ca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 --2- 172.21.15.145:0/2047302219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992380 0x559374995f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f39037fe700 0 client.26086 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:38.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 >> 172.21.15.145:0/2047302219 conn(0x5593749010e0 msgr2=0x559374981b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 shutdown_connections 2023-08-25T04:54:38.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 1 -- 172.21.15.145:0/2047302219 wait complete. 2023-08-25T04:54:38.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.327+0000 7f393de22b80 20 client.26086 trim_cache size 0 max 16384 2023-08-25T04:54:38.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.346+0000 7f393de22b80 1 Processor -- start 2023-08-25T04:54:38.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.346+0000 7f393de22b80 1 -- start start 2023-08-25T04:54:38.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.346+0000 7f393de22b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x559374998840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.346+0000 7f393de22b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.346+0000 7f393de22b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374995c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f393de22b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559374760920 con 0x55937498cc40 2023-08-25T04:54:38.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f393de22b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559374987f00 con 0x559374992ca0 2023-08-25T04:54:38.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f393de22b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593749944d0 con 0x559374992380 2023-08-25T04:54:38.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f3923fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x559374998840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35882/0 (socket says 172.21.15.145:35882) 2023-08-25T04:54:38.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 -- 172.21.15.145:0/2806526702 learned_addr learned my addr 172.21.15.145:0/2806526702 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:38.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f392899c700 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374995c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 -- 172.21.15.145:0/2806526702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 msgr2=0x559374995c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374995c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 -- 172.21.15.145:0/2806526702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 msgr2=0x559374998840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x559374998840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 -- 172.21.15.145:0/2806526702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593746f3c60 con 0x559374992380 2023-08-25T04:54:38.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.347+0000 7f39237fe700 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3918098e10 tx=0x7f3918010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7614c850c6f81b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.348+0000 7f390affd700 1 -- 172.21.15.145:0/2806526702 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180b1020 con 0x559374992380 2023-08-25T04:54:38.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.348+0000 7f390affd700 1 -- 172.21.15.145:0/2806526702 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3918099e90 con 0x559374992380 2023-08-25T04:54:38.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.348+0000 7f390affd700 1 -- 172.21.15.145:0/2806526702 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180985d0 con 0x559374992380 2023-08-25T04:54:38.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.348+0000 7f393de22b80 1 -- 172.21.15.145:0/2806526702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 msgr2=0x559374995730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.348+0000 7f393de22b80 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3918098e10 tx=0x7f3918010c60 comp rx=0 tx=0).stop 2023-08-25T04:54:38.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 -- 172.21.15.145:0/2806526702 shutdown_connections 2023-08-25T04:54:38.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374995c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x559374998840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 --2- 172.21.15.145:0/2806526702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374995730 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 -- 172.21.15.145:0/2806526702 >> 172.21.15.145:0/2806526702 conn(0x5593749010e0 msgr2=0x5593749900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 -- 172.21.15.145:0/2806526702 shutdown_connections 2023-08-25T04:54:38.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 -- 172.21.15.145:0/2806526702 wait complete. 2023-08-25T04:54:38.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 -- start start 2023-08-25T04:54:38.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.349+0000 7f393de22b80 1 Processor -- start 2023-08-25T04:54:38.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937499d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374999d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937499a290 con 0x55937498cc40 2023-08-25T04:54:38.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937499a3d0 con 0x559374992ca0 2023-08-25T04:54:38.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937499a590 con 0x559374992380 2023-08-25T04:54:38.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f393de22b80 10 client.0 _ll_register_callbacks cb 0x7f3914005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:54:38.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f39237fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35898/0 (socket says 172.21.15.145:35898) 2023-08-25T04:54:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f39237fe700 1 -- 172.21.15.145:0/2305982328 learned_addr learned my addr 172.21.15.145:0/2305982328 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:54:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f3923fff700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937499d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f392899c700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374999d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f39237fe700 1 -- 172.21.15.145:0/2305982328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 msgr2=0x559374999d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.350+0000 7f39237fe700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374999d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39237fe700 1 -- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 msgr2=0x55937499d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:54:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39237fe700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937499d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39237fe700 1 -- 172.21.15.145:0/2305982328 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55937475e630 con 0x559374992380 2023-08-25T04:54:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f3923fff700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937499d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:54:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39237fe700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f391809ce70 tx=0x7f39180b10f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=446844aba148e8d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:54:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39037fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:54:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180b8200 con 0x559374992380 2023-08-25T04:54:38.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f39180990e0 con 0x559374992380 2023-08-25T04:54:38.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55937497f8b0 con 0x559374992380 2023-08-25T04:54:38.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39180b8360 con 0x559374992380 2023-08-25T04:54:38.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f393de22b80 10 client.26092 fetch_fsmap learned FSMap version 35 2023-08-25T04:54:38.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559374731250 con 0x559374992380 2023-08-25T04:54:38.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.351+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55937497f8b0 con 0x559374992380 2023-08-25T04:54:38.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39180a0920 con 0x559374992380 2023-08-25T04:54:38.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 10 client.26092 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:54:38.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:54:38.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 20 client.26092 populate_metadata read hostname 'smithi145' 2023-08-25T04:54:38.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749fb550 0x5593749fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:54:38.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 4 client.26092 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:54:38.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5593748d7970 con 0x5593749fb550 2023-08-25T04:54:38.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.352+0000 7f3923fff700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749fb550 0x5593749fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:54:38.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f3923fff700 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749fb550 0x5593749fd940 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05a8e1a50e86bd7 server_cookie=a025d75ba826c469 in_seq=0 out_seq=0 2023-08-25T04:54:38.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f39037fe700 10 client.26092 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:38.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f39037fe700 1 -- 172.21.15.145:0/2305982328 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5593748d7970 con 0x5593749fb550 2023-08-25T04:54:38.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f39037fe700 10 client.26092 handle_command_reply: tid=0 2023-08-25T04:54:38.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 client.26092 shutdown 2023-08-25T04:54:38.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 msgr2=0x559374999810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f391809ce70 tx=0x7f39180b10f0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 shutdown_connections 2023-08-25T04:54:38.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559374992ca0 0x559374999d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55937498cc40 0x55937499d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5593749fb550 0x5593749fd940 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 --2- 172.21.15.145:0/2305982328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559374992380 0x559374999810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 >> 172.21.15.145:0/2305982328 conn(0x5593749010e0 msgr2=0x5593749812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f39037fe700 0 client.26092 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:54:38.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 shutdown_connections 2023-08-25T04:54:38.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.353+0000 7f393de22b80 1 -- 172.21.15.145:0/2305982328 wait complete. 2023-08-25T04:54:38.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f393de22b80 20 client.26092 trim_cache size 0 max 16384 2023-08-25T04:54:38.380 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:54:38.381 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:54:38.381 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:54:38.381 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:54:38.381 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:54:38.381 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:54:38.382 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:54:38.382 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:54:38.382 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:54:38.382 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:54:38.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 msgr2=0x7f390c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.384 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 0x7f390c0a2580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f391400b2e0 tx=0x7f3914005fd0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 msgr2=0x7f392419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:54:38.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 0x7f392419ef30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f391800e9f0 tx=0x7f391800ed00 comp rx=0 tx=0).stop 2023-08-25T04:54:38.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 shutdown_connections 2023-08-25T04:54:38.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3924106880 0x7f3924198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.385 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f390c0a00c0 0x7f390c0a2580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924100870 0x7f392419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 --2- 172.21.15.145:0/3560641637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3924101230 0x7f392419ef30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:54:38.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 >> 172.21.15.145:0/3560641637 conn(0x7f39240fc3a0 msgr2=0x7f392410ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:54:38.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 shutdown_connections 2023-08-25T04:54:38.386 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:54:38.354+0000 7f392a3ff700 1 -- 172.21.15.145:0/3560641637 wait complete. 2023-08-25T04:54:38.387 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:54:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:39 smithi162 ceph-mon[130941]: pgmap v296: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:39 smithi183 ceph-mon[131414]: pgmap v296: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:39 smithi145 ceph-mon[97681]: pgmap v296: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:41 smithi162 ceph-mon[130941]: pgmap v297: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:41 smithi183 ceph-mon[131414]: pgmap v297: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:41 smithi145 ceph-mon[97681]: pgmap v297: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:43 smithi162 ceph-mon[130941]: pgmap v298: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:43 smithi183 ceph-mon[131414]: pgmap v298: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:43 smithi145 ceph-mon[97681]: pgmap v298: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:46.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:45 smithi162 ceph-mon[130941]: pgmap v299: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:46.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:45 smithi183 ceph-mon[131414]: pgmap v299: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:46.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:45 smithi145 ceph-mon[97681]: pgmap v299: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:48.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:47 smithi162 ceph-mon[130941]: pgmap v300: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:48.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:47 smithi183 ceph-mon[131414]: pgmap v300: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:48.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:47 smithi145 ceph-mon[97681]: pgmap v300: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:50.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:49 smithi162 ceph-mon[130941]: pgmap v301: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:49 smithi183 ceph-mon[131414]: pgmap v301: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:50.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:49 smithi145 ceph-mon[97681]: pgmap v301: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:51 smithi162 ceph-mon[130941]: pgmap v302: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:51 smithi183 ceph-mon[131414]: pgmap v302: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:52.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:51 smithi145 ceph-mon[97681]: pgmap v302: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:53 smithi162 ceph-mon[130941]: pgmap v303: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:53 smithi183 ceph-mon[131414]: pgmap v303: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:54.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:53 smithi145 ceph-mon[97681]: pgmap v303: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:55 smithi162 ceph-mon[130941]: pgmap v304: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:55 smithi183 ceph-mon[131414]: pgmap v304: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:55 smithi145 ceph-mon[97681]: pgmap v304: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:54:58.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:57 smithi162 ceph-mon[130941]: pgmap v305: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:57 smithi183 ceph-mon[131414]: pgmap v305: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:54:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:57 smithi145 ceph-mon[97681]: pgmap v305: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:54:59 smithi162 ceph-mon[130941]: pgmap v306: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:54:59 smithi183 ceph-mon[131414]: pgmap v306: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:00.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:54:59 smithi145 ceph-mon[97681]: pgmap v306: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:01 smithi162 ceph-mon[130941]: pgmap v307: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:01 smithi183 ceph-mon[131414]: pgmap v307: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:01 smithi145 ceph-mon[97681]: pgmap v307: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:03 smithi162 ceph-mon[130941]: pgmap v308: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:03 smithi183 ceph-mon[131414]: pgmap v308: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:03 smithi145 ceph-mon[97681]: pgmap v308: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:06.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:05 smithi162 ceph-mon[130941]: pgmap v309: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:05 smithi183 ceph-mon[131414]: pgmap v309: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:05 smithi145 ceph-mon[97681]: pgmap v309: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:07.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-08-25T04:55:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 -- 172.21.15.145:0/3412946527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c102ac0 msgr2=0x7f292c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 --2- 172.21.15.145:0/3412946527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c102ac0 0x7f292c102f40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f291c009990 tx=0x7f291c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 -- 172.21.15.145:0/3412946527 shutdown_connections 2023-08-25T04:55:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 --2- 172.21.15.145:0/3412946527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c103480 0x7f292c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 --2- 172.21.15.145:0/3412946527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 0x7f292c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 --2- 172.21.15.145:0/3412946527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c102ac0 0x7f292c102f40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 -- 172.21.15.145:0/3412946527 >> 172.21.15.145:0/3412946527 conn(0x7f292c0fe570 msgr2=0x7f292c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 -- 172.21.15.145:0/3412946527 shutdown_connections 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.056+0000 7f29345ff700 1 -- 172.21.15.145:0/3412946527 wait complete. 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.057+0000 7f29345ff700 1 Processor -- start 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 -- start start 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 0x7f292c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c199090 con 0x7f292c102ac0 2023-08-25T04:55:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c199200 con 0x7f292c108ad0 2023-08-25T04:55:08.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f29345ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c1993c0 con 0x7f292c103480 2023-08-25T04:55:08.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f293239b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f2931b9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f2931b9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57388/0 (socket says 172.21.15.145:57388) 2023-08-25T04:55:08.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f2931b9a700 1 -- 172.21.15.145:0/3987009830 learned_addr learned my addr 172.21.15.145:0/3987009830 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.058+0000 7f293239b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42442/0 (socket says 172.21.15.145:42442) 2023-08-25T04:55:08.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2932b9c700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 0x7f292c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 -- 172.21.15.145:0/3987009830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 msgr2=0x7f292c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 0x7f292c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 -- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 msgr2=0x7f292c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 -- 172.21.15.145:0/3987009830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0095f0 con 0x7f292c103480 2023-08-25T04:55:08.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f293239b700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:55:08.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f2931b9a700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f291c005910 tx=0x7f291c004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c01d070 con 0x7f292c103480 2023-08-25T04:55:08.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291c022470 con 0x7f292c103480 2023-08-25T04:55:08.068 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c017650 con 0x7f292c103480 2023-08-25T04:55:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.059+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292c199640 con 0x7f292c103480 2023-08-25T04:55:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.060+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292c1a5bb0 con 0x7f292c103480 2023-08-25T04:55:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.061+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292c102200 con 0x7f292c103480 2023-08-25T04:55:08.069 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.063+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f291c0225e0 con 0x7f292c103480 2023-08-25T04:55:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.064+0000 7f29237fe700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 0x7f29180a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.064+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f291c0ccbe0 con 0x7f292c103480 2023-08-25T04:55:08.070 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.065+0000 7f293239b700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 0x7f29180a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.065+0000 7f293239b700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 0x7f29180a2600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2928009fd0 tx=0x7f2928009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.066+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f291c096320 con 0x7f292c103480 2023-08-25T04:55:08.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:07 smithi145 ceph-mon[97681]: pgmap v310: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:55:08.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:07 smithi162 ceph-mon[130941]: pgmap v310: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:55:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:07 smithi183 ceph-mon[131414]: pgmap v310: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:55:08.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.356+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f292c112b30 con 0x7f29180a0140 2023-08-25T04:55:08.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.364+0000 7f29237fe700 1 -- 172.21.15.145:0/3987009830 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f292c112b30 con 0x7f29180a0140 2023-08-25T04:55:08.365 INFO:teuthology.orchestra.run.smithi145.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-08-25T00:00:00", "created": "2023-08-25T04:45:05", "first": "2023-08-25T04:46:00", "last": "2023-08-25T04:55:00", "last_pruned": "2023-08-25T04:55:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-08-25T04:55:08.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 msgr2=0x7f29180a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 0x7f29180a2600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2928009fd0 tx=0x7f2928009260 comp rx=0 tx=0).stop 2023-08-25T04:55:08.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 msgr2=0x7f292c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f291c005910 tx=0x7f291c004710 comp rx=0 tx=0).stop 2023-08-25T04:55:08.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 shutdown_connections 2023-08-25T04:55:08.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f29180a0140 0x7f29180a2600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f292c108ad0 0x7f292c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f292c102ac0 0x7f292c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 --2- 172.21.15.145:0/3987009830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f292c103480 0x7f292c19f100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.366+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 >> 172.21.15.145:0/3987009830 conn(0x7f292c0fe570 msgr2=0x7f292c1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.367+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 shutdown_connections 2023-08-25T04:55:08.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.367+0000 7f29345ff700 1 -- 172.21.15.145:0/3987009830 wait complete. 2023-08-25T04:55:08.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:55:08.390 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-08-25T04:55:08.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.668+0000 7f08e39ff700 1 -- 172.21.15.145:0/633112551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc1068d0 msgr2=0x7f08dc106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.668+0000 7f08e39ff700 1 --2- 172.21.15.145:0/633112551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc1068d0 0x7f08dc106cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009990 tx=0x7f08d8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 -- 172.21.15.145:0/633112551 shutdown_connections 2023-08-25T04:55:08.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 --2- 172.21.15.145:0/633112551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc10d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 --2- 172.21.15.145:0/633112551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc1068d0 0x7f08dc106cb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 --2- 172.21.15.145:0/633112551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc067d10 0x7f08dc068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 -- 172.21.15.145:0/633112551 >> 172.21.15.145:0/633112551 conn(0x7f08dc075250 msgr2=0x7f08dc075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 -- 172.21.15.145:0/633112551 shutdown_connections 2023-08-25T04:55:08.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.669+0000 7f08e39ff700 1 -- 172.21.15.145:0/633112551 wait complete. 2023-08-25T04:55:08.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.670+0000 7f08e39ff700 1 Processor -- start 2023-08-25T04:55:08.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.670+0000 7f08e39ff700 1 -- start start 2023-08-25T04:55:08.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc1068d0 0x7f08dc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19cdb0 con 0x7f08dc0686d0 2023-08-25T04:55:08.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19cf20 con 0x7f08dc1068d0 2023-08-25T04:55:08.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e39ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19d0e0 con 0x7f08dc067d10 2023-08-25T04:55:08.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e179b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e179b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57404/0 (socket says 172.21.15.145:57404) 2023-08-25T04:55:08.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e179b700 1 -- 172.21.15.145:0/3758084732 learned_addr learned my addr 172.21.15.145:0/3758084732 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e0f9a700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e1f9c700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc1068d0 0x7f08dc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e179b700 1 -- 172.21.15.145:0/3758084732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc1068d0 msgr2=0x7f08dc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.671+0000 7f08e179b700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc1068d0 0x7f08dc19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e179b700 1 -- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 msgr2=0x7f08dc1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e179b700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e179b700 1 -- 172.21.15.145:0/3758084732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80095f0 con 0x7f08dc067d10 2023-08-25T04:55:08.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e0f9a700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:55:08.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e179b700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f08d800f690 tx=0x7f08d800f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d801d070 con 0x7f08dc067d10 2023-08-25T04:55:08.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08d8005a00 con 0x7f08dc067d10 2023-08-25T04:55:08.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc19d360 con 0x7f08dc067d10 2023-08-25T04:55:08.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80056f0 con 0x7f08dc067d10 2023-08-25T04:55:08.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.672+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1a5d20 con 0x7f08dc067d10 2023-08-25T04:55:08.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.673+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc075e80 con 0x7f08dc067d10 2023-08-25T04:55:08.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.676+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f08d8003960 con 0x7f08dc067d10 2023-08-25T04:55:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.678+0000 7f08d27fc700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 0x7f08c80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.678+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f08d80ccb30 con 0x7f08dc067d10 2023-08-25T04:55:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.678+0000 7f08e0f9a700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 0x7f08c80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.679+0000 7f08e0f9a700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 0x7f08c80a2600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005950 tx=0x7f08cc0058e0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.679+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f08d80962d0 con 0x7f08dc067d10 2023-08-25T04:55:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.679+0000 7f17695ff700 1 -- 172.21.15.145:0/2078025363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176410d110 msgr2=0x7f176410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.679+0000 7f17695ff700 1 --2- 172.21.15.145:0/2078025363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176410d110 0x7f176410f540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f175c009990 tx=0x7f175c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 -- 172.21.15.145:0/2078025363 shutdown_connections 2023-08-25T04:55:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 --2- 172.21.15.145:0/2078025363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176410d110 0x7f176410f540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 --2- 172.21.15.145:0/2078025363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1764072d10 0x7f17640730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 --2- 172.21.15.145:0/2078025363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764073630 0x7f176410cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 -- 172.21.15.145:0/2078025363 >> 172.21.15.145:0/2078025363 conn(0x7f17640fc380 msgr2=0x7f17640fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 -- 172.21.15.145:0/2078025363 shutdown_connections 2023-08-25T04:55:08.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.680+0000 7f17695ff700 1 -- 172.21.15.145:0/2078025363 wait complete. 2023-08-25T04:55:08.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.681+0000 7f17695ff700 1 Processor -- start 2023-08-25T04:55:08.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 -- start start 2023-08-25T04:55:08.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764072d10 0x7f176419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176410d110 0x7f17641989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764199030 con 0x7f176410d110 2023-08-25T04:55:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641991a0 con 0x7f1764072d10 2023-08-25T04:55:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17695ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764199360 con 0x7f1764073630 2023-08-25T04:55:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57416/0 (socket says 172.21.15.145:57416) 2023-08-25T04:55:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f17627fc700 1 -- 172.21.15.145:0/3591129356 learned_addr learned my addr 172.21.15.145:0/3591129356 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.682+0000 7f1762ffd700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764072d10 0x7f176419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17637fe700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176410d110 0x7f17641989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 -- 172.21.15.145:0/3591129356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764072d10 msgr2=0x7f176419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764072d10 0x7f176419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 -- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176410d110 msgr2=0x7f17641989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176410d110 0x7f17641989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 -- 172.21.15.145:0/3591129356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0095f0 con 0x7f1764073630 2023-08-25T04:55:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17627fc700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f175800d950 tx=0x7f175800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f174bfff700 1 -- 172.21.15.145:0/3591129356 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800f890 con 0x7f1764073630 2023-08-25T04:55:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f174bfff700 1 -- 172.21.15.145:0/3591129356 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1758010460 con 0x7f1764073630 2023-08-25T04:55:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1764199640 con 0x7f1764073630 2023-08-25T04:55:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.683+0000 7f174bfff700 1 -- 172.21.15.145:0/3591129356 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800e610 con 0x7f1764073630 2023-08-25T04:55:08.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.684+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17641a5940 con 0x7f1764073630 2023-08-25T04:55:08.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.687+0000 7f174bfff700 1 -- 172.21.15.145:0/3591129356 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f175800f9f0 con 0x7f1764073630 2023-08-25T04:55:08.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.689+0000 7f174bfff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 0x7f174c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.689+0000 7f174bfff700 1 -- 172.21.15.145:0/3591129356 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f17580cb610 con 0x7f1764073630 2023-08-25T04:55:08.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.689+0000 7f1762ffd700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 0x7f174c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.690+0000 7f1762ffd700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 0x7f174c0a2600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1754009550 tx=0x7f17540091b0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.690+0000 7f177ce30b80 1 Processor -- start 2023-08-25T04:55:08.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.691+0000 7f177ce30b80 1 -- start start 2023-08-25T04:55:08.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.691+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.691+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.691+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f177ce30b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a7087b0 con 0x55fe7a7a6380 2023-08-25T04:55:08.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f177ce30b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a6f9780 con 0x55fe7a7a6ca0 2023-08-25T04:55:08.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f177ce30b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a6f70a0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57428/0 (socket says 172.21.15.145:57428) 2023-08-25T04:55:08.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 -- 172.21.15.145:0/2926892761 learned_addr learned my addr 172.21.15.145:0/2926892761 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f1762ffd700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17627fc700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 -- 172.21.15.145:0/2926892761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 msgr2=0x55fe7a7a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 -- 172.21.15.145:0/2926892761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 msgr2=0x55fe7a7a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 -- 172.21.15.145:0/2926892761 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe7a572630 con 0x55fe7a7a0c40 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f1762ffd700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.692+0000 7f17637fe700 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f175c005e80 tx=0x7f175c009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22c87d62a67cee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.693+0000 7f17497fa700 1 -- 172.21.15.145:0/2926892761 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c01d050 con 0x55fe7a7a0c40 2023-08-25T04:55:08.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.693+0000 7f17497fa700 1 -- 172.21.15.145:0/2926892761 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c01e040 con 0x55fe7a7a0c40 2023-08-25T04:55:08.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.693+0000 7f17497fa700 1 -- 172.21.15.145:0/2926892761 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c023de0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.693+0000 7f177ce30b80 1 -- 172.21.15.145:0/2926892761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 msgr2=0x55fe7a7a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.693+0000 7f177ce30b80 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f175c005e80 tx=0x7f175c009ee0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 -- 172.21.15.145:0/2926892761 shutdown_connections 2023-08-25T04:55:08.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a10a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 --2- 172.21.15.145:0/2926892761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 -- 172.21.15.145:0/2926892761 >> 172.21.15.145:0/2926892761 conn(0x55fe7a7150e0 msgr2=0x55fe7a79b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 -- 172.21.15.145:0/2926892761 shutdown_connections 2023-08-25T04:55:08.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 -- 172.21.15.145:0/2926892761 wait complete. 2023-08-25T04:55:08.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.694+0000 7f177ce30b80 1 -- start start 2023-08-25T04:55:08.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 Processor -- start 2023-08-25T04:55:08.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a6f2990 con 0x55fe7a7a0c40 2023-08-25T04:55:08.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a690bb0 con 0x55fe7a7a6ca0 2023-08-25T04:55:08.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a6908b0 con 0x55fe7a7a6380 2023-08-25T04:55:08.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.695+0000 7f177ce30b80 10 client.0 _ll_register_callbacks cb 0x55fe7a5daa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:55:08.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17637fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57430/0 (socket says 172.21.15.145:57430) 2023-08-25T04:55:08.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17637fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:51398/0 (socket says 172.21.15.145:51398) 2023-08-25T04:55:08.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 -- 172.21.15.145:0/942529322 learned_addr learned my addr 172.21.15.145:0/942529322 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f1762ffd700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 -- 172.21.15.145:0/942529322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 msgr2=0x55fe7a7aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 -- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 msgr2=0x55fe7a7acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 -- 172.21.15.145:0/942529322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1764110200 con 0x55fe7a7a6380 2023-08-25T04:55:08.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f1762ffd700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:55:08.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f17627fc700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f175800b2f0 tx=0x7f17580989e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d1cb265d9addb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f1746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:55:08.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580108d0 con 0x55fe7a7a6380 2023-08-25T04:55:08.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f17580950f0 con 0x55fe7a7a6380 2023-08-25T04:55:08.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.696+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe7a7701c0 con 0x55fe7a7a6380 2023-08-25T04:55:08.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1758010a30 con 0x55fe7a7a6380 2023-08-25T04:55:08.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 10 client.26116 fetch_fsmap learned FSMap version 35 2023-08-25T04:55:08.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe7a51ede0 con 0x55fe7a7a6380 2023-08-25T04:55:08.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe7a7701c0 con 0x55fe7a7a6380 2023-08-25T04:55:08.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1758098c40 con 0x55fe7a7a6380 2023-08-25T04:55:08.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:55:08.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:55:08.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 20 client.26116 populate_metadata read hostname 'smithi145' 2023-08-25T04:55:08.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a7ba5a0 0x55fe7a7bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 4 client.26116 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:55:08.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe7a698ba0 con 0x55fe7a7ba5a0 2023-08-25T04:55:08.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.697+0000 7f1762ffd700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a7ba5a0 0x55fe7a7bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.698+0000 7f1746ff5700 10 client.26116 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:08.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.698+0000 7f1762ffd700 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a7ba5a0 0x55fe7a7bc9c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781afe6d080115f3 server_cookie=eb0e69d152544ca8 in_seq=0 out_seq=0 2023-08-25T04:55:08.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f1746ff5700 1 -- 172.21.15.145:0/942529322 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe7a698ba0 con 0x55fe7a7ba5a0 2023-08-25T04:55:08.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f1746ff5700 10 client.26116 handle_command_reply: tid=0 2023-08-25T04:55:08.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 client.26116 shutdown 2023-08-25T04:55:08.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 msgr2=0x55fe7a7a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f175800b2f0 tx=0x7f17580989e0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 shutdown_connections 2023-08-25T04:55:08.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a7ba5a0 0x55fe7a7bc9c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 --2- 172.21.15.145:0/942529322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a9e80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 >> 172.21.15.145:0/942529322 conn(0x55fe7a7150e0 msgr2=0x55fe7a795b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f1746ff5700 0 client.26116 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.699+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 shutdown_connections 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.700+0000 7f177ce30b80 1 -- 172.21.15.145:0/942529322 wait complete. 2023-08-25T04:55:08.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.700+0000 7f177ce30b80 20 client.26116 trim_cache size 0 max 16384 2023-08-25T04:55:08.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.724+0000 7f177ce30b80 1 Processor -- start 2023-08-25T04:55:08.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.724+0000 7f177ce30b80 1 -- start start 2023-08-25T04:55:08.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a574920 con 0x55fe7a7a6ca0 2023-08-25T04:55:08.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a7ab460 con 0x55fe7a7a0c40 2023-08-25T04:55:08.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f177ce30b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a797c80 con 0x55fe7a7a6380 2023-08-25T04:55:08.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57446/0 (socket says 172.21.15.145:57446) 2023-08-25T04:55:08.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 -- 172.21.15.145:0/3782277601 learned_addr learned my addr 172.21.15.145:0/3782277601 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17637fe700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f1762ffd700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 -- 172.21.15.145:0/3782277601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a0c40 msgr2=0x55fe7a7a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 -- 172.21.15.145:0/3782277601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 msgr2=0x55fe7a7a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17627fc700 1 -- 172.21.15.145:0/3782277601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe7a507c60 con 0x55fe7a7a6380 2023-08-25T04:55:08.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.725+0000 7f17637fe700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:55:08.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.726+0000 7f17627fc700 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17580c8fa0 tx=0x7f1758000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a95a85ba46ad00d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.726+0000 7f1749ffb700 1 -- 172.21.15.145:0/3782277601 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800fca0 con 0x55fe7a7a6380 2023-08-25T04:55:08.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.726+0000 7f1749ffb700 1 -- 172.21.15.145:0/3782277601 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17580108d0 con 0x55fe7a7a6380 2023-08-25T04:55:08.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.726+0000 7f1749ffb700 1 -- 172.21.15.145:0/3782277601 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580cb280 con 0x55fe7a7a6380 2023-08-25T04:55:08.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- 172.21.15.145:0/3782277601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 msgr2=0x55fe7a7ac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17580c8fa0 tx=0x7f1758000bc0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- 172.21.15.145:0/3782277601 shutdown_connections 2023-08-25T04:55:08.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.738 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 --2- 172.21.15.145:0/3782277601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a6380 0x55fe7a7ac7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- 172.21.15.145:0/3782277601 >> 172.21.15.145:0/3782277601 conn(0x55fe7a7150e0 msgr2=0x55fe7a7a4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- 172.21.15.145:0/3782277601 shutdown_connections 2023-08-25T04:55:08.739 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- 172.21.15.145:0/3782277601 wait complete. 2023-08-25T04:55:08.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.727+0000 7f177ce30b80 1 -- start start 2023-08-25T04:55:08.740 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 Processor -- start 2023-08-25T04:55:08.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.741 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a7a2c50 con 0x55fe7a7a6ca0 2023-08-25T04:55:08.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a7a2dc0 con 0x55fe7a7a6380 2023-08-25T04:55:08.742 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe7a80e3f0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f177ce30b80 10 client.0 _ll_register_callbacks cb 0x7f1758098340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:55:08.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f1762ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.743 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.728+0000 7f1762ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57462/0 (socket says 172.21.15.145:57462) 2023-08-25T04:55:08.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 -- 172.21.15.145:0/1672050331 learned_addr learned my addr 172.21.15.145:0/1672050331 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:08.744 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f17627fc700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 -- 172.21.15.145:0/1672050331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 msgr2=0x55fe7a7a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f17637fe700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.745 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 -- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 msgr2=0x55fe7a7a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:08.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.746 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 -- 172.21.15.145:0/1672050331 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe7a51ede0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f17627fc700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:55:08.747 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f17637fe700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:55:08.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1762ffd700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55fe7a7bd2c0 tx=0x7f1754004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3dcf69f702f846 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:08.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:55:08.748 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.729+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754015850 con 0x55fe7a7a0c40 2023-08-25T04:55:08.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f17540159b0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.749 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe7a7938b0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1754007e50 con 0x55fe7a7a0c40 2023-08-25T04:55:08.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 10 client.26122 fetch_fsmap learned FSMap version 35 2023-08-25T04:55:08.750 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe7a545250 con 0x55fe7a7a0c40 2023-08-25T04:55:08.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe7a7938b0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17540163f0 con 0x55fe7a7a0c40 2023-08-25T04:55:08.751 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:55:08.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:55:08.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 20 client.26122 populate_metadata read hostname 'smithi145' 2023-08-25T04:55:08.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a80fa00 0x55fe7a811e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:08.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 4 client.26122 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:55:08.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.730+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe7a6eb970 con 0x55fe7a80fa00 2023-08-25T04:55:08.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f17627fc700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a80fa00 0x55fe7a811e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:08.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f17627fc700 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a80fa00 0x55fe7a811e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92fc4fa099b08835 server_cookie=384570384c8016da in_seq=0 out_seq=0 2023-08-25T04:55:08.754 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f1746ff5700 10 client.26122 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:08.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f1746ff5700 1 -- 172.21.15.145:0/1672050331 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55fe7a6eb970 con 0x55fe7a80fa00 2023-08-25T04:55:08.755 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f1746ff5700 10 client.26122 handle_command_reply: tid=0 2023-08-25T04:55:08.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.731+0000 7f177ce30b80 1 client.26122 shutdown 2023-08-25T04:55:08.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 msgr2=0x55fe7a7b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55fe7a7bd2c0 tx=0x7f1754004b40 comp rx=0 tx=0).stop 2023-08-25T04:55:08.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 shutdown_connections 2023-08-25T04:55:08.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fe7a7a6ca0 0x55fe7a7a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55fe7a80fa00 0x55fe7a811e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe7a7a0c40 0x55fe7a7b1740 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 --2- 172.21.15.145:0/1672050331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fe7a7a6380 0x55fe7a7a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f1746ff5700 0 client.26122 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:08.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 >> 172.21.15.145:0/1672050331 conn(0x55fe7a7150e0 msgr2=0x55fe7a7952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 shutdown_connections 2023-08-25T04:55:08.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 1 -- 172.21.15.145:0/1672050331 wait complete. 2023-08-25T04:55:08.760 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.732+0000 7f177ce30b80 20 client.26122 trim_cache size 0 max 16384 2023-08-25T04:55:08.761 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:55:08.762 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:55:08.762 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:55:08.762 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:55:08.763 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:55:08.763 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:55:08.764 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:55:08.764 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:55:08.764 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:55:08.764 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:55:08.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 msgr2=0x7f174c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 0x7f174c0a2600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1754009550 tx=0x7f17540091b0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 msgr2=0x7f176419ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f175800d950 tx=0x7f175800dc60 comp rx=0 tx=0).stop 2023-08-25T04:55:08.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 shutdown_connections 2023-08-25T04:55:08.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176410d110 0x7f17641989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f174c0a0140 0x7f174c0a2600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1764072d10 0x7f176419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 --2- 172.21.15.145:0/3591129356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1764073630 0x7f176419ee90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 >> 172.21.15.145:0/3591129356 conn(0x7f17640fc380 msgr2=0x7f1764104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.767 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 shutdown_connections 2023-08-25T04:55:08.768 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.733+0000 7f17695ff700 1 -- 172.21.15.145:0/3591129356 wait complete. 2023-08-25T04:55:08.768 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:55:08.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.950+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 --> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f08dc19a270 con 0x7f08c80a0140 2023-08-25T04:55:08.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.958+0000 7f08d27fc700 1 -- 172.21.15.145:0/3758084732 <== mgr.15495 v2:172.21.15.145:6800/2587440661 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f08dc19a270 con 0x7f08c80a0140 2023-08-25T04:55:08.959 INFO:teuthology.orchestra.run.smithi145.stdout:/ 1M 6M3h 2023-08-25T04:55:08.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 msgr2=0x7f08c80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 0x7f08c80a2600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005950 tx=0x7f08cc0058e0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 msgr2=0x7f08dc198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:08.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f08d800f690 tx=0x7f08d800f770 comp rx=0 tx=0).stop 2023-08-25T04:55:08.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 shutdown_connections 2023-08-25T04:55:08.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f08c80a0140 0x7f08c80a2600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc1068d0 0x7f08dc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08dc067d10 0x7f08dc198cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 --2- 172.21.15.145:0/3758084732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08dc0686d0 0x7f08dc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:08.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 >> 172.21.15.145:0/3758084732 conn(0x7f08dc075250 msgr2=0x7f08dc10bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:08.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 shutdown_connections 2023-08-25T04:55:08.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:08.960+0000 7f08e39ff700 1 -- 172.21.15.145:0/3758084732 wait complete. 2023-08-25T04:55:08.982 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-08-25T04:55:08.992 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-08-25T04:55:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:09 smithi162 ceph-mon[130941]: from='client.26098 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:10.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:09 smithi162 ceph-mon[130941]: pgmap v311: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:55:10.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:09 smithi162 ceph-mon[130941]: from='client.26104 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:09 smithi183 ceph-mon[131414]: from='client.26098 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:09 smithi183 ceph-mon[131414]: pgmap v311: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:55:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:09 smithi183 ceph-mon[131414]: from='client.26104 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:09 smithi145 ceph-mon[97681]: from='client.26098 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:09 smithi145 ceph-mon[97681]: pgmap v311: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:55:10.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:09 smithi145 ceph-mon[97681]: from='client.26104 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-25T04:55:12.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:11 smithi162 ceph-mon[130941]: pgmap v312: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-08-25T04:55:12.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:11 smithi162 ceph-mon[130941]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T04:55:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:11 smithi183 ceph-mon[131414]: pgmap v312: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-08-25T04:55:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:11 smithi183 ceph-mon[131414]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T04:55:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:11 smithi145 ceph-mon[97681]: pgmap v312: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-08-25T04:55:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:11 smithi145 ceph-mon[97681]: osdmap e97: 12 total, 12 up, 12 in 2023-08-25T04:55:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:13 smithi162 ceph-mon[130941]: pgmap v314: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:13 smithi183 ceph-mon[131414]: pgmap v314: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:13 smithi145 ceph-mon[97681]: pgmap v314: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:15 smithi183 ceph-mon[131414]: pgmap v315: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:16.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:15 smithi145 ceph-mon[97681]: pgmap v315: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:16.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:15 smithi162 ceph-mon[130941]: pgmap v315: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-08-25T04:55:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:17 smithi183 ceph-mon[131414]: pgmap v316: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:17 smithi145 ceph-mon[97681]: pgmap v316: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:18.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:17 smithi162 ceph-mon[130941]: pgmap v316: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:19 smithi183 ceph-mon[131414]: pgmap v317: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:20.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:19 smithi183 ceph-mon[131414]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T04:55:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:19 smithi145 ceph-mon[97681]: pgmap v317: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:19 smithi145 ceph-mon[97681]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T04:55:20.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:19 smithi162 ceph-mon[130941]: pgmap v317: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2023-08-25T04:55:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:19 smithi162 ceph-mon[130941]: osdmap e98: 12 total, 12 up, 12 in 2023-08-25T04:55:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:21 smithi183 ceph-mon[131414]: pgmap v319: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-08-25T04:55:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:21 smithi145 ceph-mon[97681]: pgmap v319: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-08-25T04:55:22.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:21 smithi162 ceph-mon[130941]: pgmap v319: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-08-25T04:55:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:23 smithi145 ceph-mon[97681]: pgmap v320: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:24.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:23 smithi162 ceph-mon[130941]: pgmap v320: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:24.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:23 smithi183 ceph-mon[131414]: pgmap v320: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:25.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:24 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:55:25.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:24 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:55:25.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:24 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:55:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:25 smithi145 ceph-mon[97681]: pgmap v321: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:26.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:25 smithi162 ceph-mon[130941]: pgmap v321: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:26.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:25 smithi183 ceph-mon[131414]: pgmap v321: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:55:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: pgmap v322: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:27 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: pgmap v322: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:28.970 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.971 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.971 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.972 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.972 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.973 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.973 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.974 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.974 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.974 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.975 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.975 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:27 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.976 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: pgmap v322: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:28.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:28.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:28.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:27 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:29.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:55:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:55:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:55:29.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:55:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:29.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:55:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:55:29.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:55:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:55:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:29.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:29.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi162 to 196.2M 2023-08-25T04:55:29.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi162 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-08-25T04:55:29.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi183 to 196.2M 2023-08-25T04:55:29.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi183 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-08-25T04:55:29.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:29.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:30.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: pgmap v323: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:55:30.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:55:30.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:29 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:30.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: pgmap v323: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:30.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:55:30.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:55:30.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:29 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:30.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: pgmap v323: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:55:30.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-25T04:55:30.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:55:30.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:55:30.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:29 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:55:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:30 smithi145 ceph-mon[97681]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:55:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:30 smithi145 ceph-mon[97681]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:55:31.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:30 smithi162 ceph-mon[130941]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:55:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:30 smithi162 ceph-mon[130941]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:55:31.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:30 smithi183 ceph-mon[131414]: Adjusting osd_memory_target on smithi145 to 220.9M 2023-08-25T04:55:31.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:30 smithi183 ceph-mon[131414]: Unable to set osd_memory_target on smithi145 to 231659008: error parsing value: Value '231659008' is below minimum 939524096 2023-08-25T04:55:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:31 smithi145 ceph-mon[97681]: pgmap v324: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:55:32.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:31 smithi162 ceph-mon[130941]: pgmap v324: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:55:32.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:31 smithi183 ceph-mon[131414]: pgmap v324: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:55:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:33 smithi145 ceph-mon[97681]: pgmap v325: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:34.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:33 smithi162 ceph-mon[130941]: pgmap v325: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:34.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:33 smithi183 ceph-mon[131414]: pgmap v325: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:35 smithi145 ceph-mon[97681]: pgmap v326: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:35 smithi162 ceph-mon[130941]: pgmap v326: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:35 smithi183 ceph-mon[131414]: pgmap v326: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:38.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:37 smithi145 ceph-mon[97681]: pgmap v327: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:37 smithi162 ceph-mon[130941]: pgmap v327: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:37 smithi183 ceph-mon[131414]: pgmap v327: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:38.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:55:39.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 -- 172.21.15.145:0/3276589769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 msgr2=0x7f719c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 --2- 172.21.15.145:0/3276589769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c100b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f718c009990 tx=0x7f718c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 -- 172.21.15.145:0/3276589769 shutdown_connections 2023-08-25T04:55:39.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 --2- 172.21.15.145:0/3276589769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c101070 0x7f719c10d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 --2- 172.21.15.145:0/3276589769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c1066c0 0x7f719c106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 --2- 172.21.15.145:0/3276589769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c100b30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 -- 172.21.15.145:0/3276589769 >> 172.21.15.145:0/3276589769 conn(0x7f719c0fc160 msgr2=0x7f719c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 -- 172.21.15.145:0/3276589769 shutdown_connections 2023-08-25T04:55:39.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.003+0000 7f71a279d700 1 -- 172.21.15.145:0/3276589769 wait complete. 2023-08-25T04:55:39.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.004+0000 7f71a279d700 1 Processor -- start 2023-08-25T04:55:39.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.004+0000 7f71a279d700 1 -- start start 2023-08-25T04:55:39.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c101070 0x7f719c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c1066c0 0x7f719c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c198d80 con 0x7f719c101070 2023-08-25T04:55:39.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c198ef0 con 0x7f719c1066c0 2023-08-25T04:55:39.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a279d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1990b0 con 0x7f719c1006b0 2023-08-25T04:55:39.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a179b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a179b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43916/0 (socket says 172.21.15.145:43916) 2023-08-25T04:55:39.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a179b700 1 -- 172.21.15.145:0/3950035318 learned_addr learned my addr 172.21.15.145:0/3950035318 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:39.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c101070 0x7f719c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.005+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c1066c0 0x7f719c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 -- 172.21.15.145:0/3950035318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c1066c0 msgr2=0x7f719c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c1066c0 0x7f719c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 -- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c101070 msgr2=0x7f719c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c101070 0x7f719c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 -- 172.21.15.145:0/3950035318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198009520 con 0x7f719c1006b0 2023-08-25T04:55:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a179b700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f719800eb30 tx=0x7f719800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71927fc700 1 -- 172.21.15.145:0/3950035318 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198009880 con 0x7f719c1006b0 2023-08-25T04:55:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71927fc700 1 -- 172.21.15.145:0/3950035318 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71980044e0 con 0x7f719c1006b0 2023-08-25T04:55:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c0095f0 con 0x7f719c1006b0 2023-08-25T04:55:39.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71927fc700 1 -- 172.21.15.145:0/3950035318 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198003d80 con 0x7f719c1006b0 2023-08-25T04:55:39.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.006+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c199580 con 0x7f719c1006b0 2023-08-25T04:55:39.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.009+0000 7f71927fc700 1 -- 172.21.15.145:0/3950035318 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f7198003f20 con 0x7f719c1006b0 2023-08-25T04:55:39.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 Processor -- start 2023-08-25T04:55:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 -- start start 2023-08-25T04:55:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71927fc700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 0x7f71880a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22c537b0 con 0x556a22cf1500 2023-08-25T04:55:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22c44780 con 0x556a22cebc40 2023-08-25T04:55:39.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.011+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22c420a0 con 0x556a22cf1e20 2023-08-25T04:55:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71927fc700 1 -- 172.21.15.145:0/3950035318 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f7198014070 con 0x7f719c1006b0 2023-08-25T04:55:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43930/0 (socket says 172.21.15.145:43930) 2023-08-25T04:55:39.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 -- 172.21.15.145:0/1122957457 learned_addr learned my addr 172.21.15.145:0/1122957457 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a179b700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 0x7f71880a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 -- 172.21.15.145:0/1122957457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 msgr2=0x556a22cec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cec080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 -- 172.21.15.145:0/1122957457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 msgr2=0x556a22cf18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 -- 172.21.15.145:0/1122957457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a22abd630 con 0x556a22cf1e20 2023-08-25T04:55:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:55:39.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.012+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f718c015000 tx=0x7f718c005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714ca152f6b5d774 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71877fe700 1 -- 172.21.15.145:0/1122957457 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c01d070 con 0x556a22cf1e20 2023-08-25T04:55:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71877fe700 1 -- 172.21.15.145:0/1122957457 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c01e040 con 0x556a22cf1e20 2023-08-25T04:55:39.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71877fe700 1 -- 172.21.15.145:0/1122957457 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c0048f0 con 0x556a22cf1e20 2023-08-25T04:55:39.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71a179b700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 0x7f71880a26a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71980c90a0 tx=0x7f7198009f60 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71b73a0b80 1 -- 172.21.15.145:0/1122957457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 msgr2=0x556a22ceb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.013+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f718c015000 tx=0x7f718c005ea0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.029 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 -- 172.21.15.145:0/1122957457 shutdown_connections 2023-08-25T04:55:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/1122957457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22ceb700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 -- 172.21.15.145:0/1122957457 >> 172.21.15.145:0/1122957457 conn(0x556a22c600e0 msgr2=0x556a22ce6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.030 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 -- 172.21.15.145:0/1122957457 shutdown_connections 2023-08-25T04:55:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 -- 172.21.15.145:0/1122957457 wait complete. 2023-08-25T04:55:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 -- start start 2023-08-25T04:55:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.014+0000 7f71b73a0b80 1 Processor -- start 2023-08-25T04:55:39.031 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cf5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22c3d990 con 0x556a22cf1500 2023-08-25T04:55:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22bdbbb0 con 0x556a22cebc40 2023-08-25T04:55:39.032 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22bdb8b0 con 0x556a22cf1e20 2023-08-25T04:55:39.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71b73a0b80 10 client.0 _ll_register_callbacks cb 0x556a22b25a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:55:39.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43938/0 (socket says 172.21.15.145:43938) 2023-08-25T04:55:39.033 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.015+0000 7f71a1f9c700 1 -- 172.21.15.145:0/2903816340 learned_addr learned my addr 172.21.15.145:0/2903816340 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cf5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 -- 172.21.15.145:0/2903816340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 msgr2=0x556a22cf5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cf5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.034 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 -- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 msgr2=0x556a22cf56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 -- 172.21.15.145:0/2903816340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7194013690 con 0x556a22cf1e20 2023-08-25T04:55:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7194009460 tx=0x7f719400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d820fdf6db823c95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f7184ff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:55:39.035 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719400e040 con 0x556a22cf1e20 2023-08-25T04:55:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a22cbb1c0 con 0x556a22cf1e20 2023-08-25T04:55:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.016+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f7194018440 con 0x556a22cf1e20 2023-08-25T04:55:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71940267a0 con 0x556a22cf1e20 2023-08-25T04:55:39.036 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 10 client.26134 fetch_fsmap learned FSMap version 35 2023-08-25T04:55:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f719c10de80 con 0x556a22cf1e20 2023-08-25T04:55:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a22cbb1c0 con 0x556a22cf1e20 2023-08-25T04:55:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7194018860 con 0x556a22cf1e20 2023-08-25T04:55:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 10 client.26134 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:55:39.037 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:55:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 20 client.26134 populate_metadata read hostname 'smithi145' 2023-08-25T04:55:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d05930 0x556a22d07d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 4 client.26134 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:55:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a22be3ba0 con 0x556a22d05930 2023-08-25T04:55:39.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.017+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d05930 0x556a22d07d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.018+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d05930 0x556a22d07d50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8010ae518e0866f7 server_cookie=8de21627b4b2faff in_seq=0 out_seq=0 2023-08-25T04:55:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.018+0000 7f7184ff9700 10 client.26134 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f7184ff9700 1 -- 172.21.15.145:0/2903816340 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a22be3ba0 con 0x556a22d05930 2023-08-25T04:55:39.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f7184ff9700 10 client.26134 handle_command_reply: tid=0 2023-08-25T04:55:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 client.26134 shutdown 2023-08-25T04:55:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 msgr2=0x556a22cf5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7194009460 tx=0x7f719400b040 comp rx=0 tx=0).stop 2023-08-25T04:55:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 shutdown_connections 2023-08-25T04:55:39.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d05930 0x556a22d07d50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1e20 0x556a22cf5bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cebc40 0x556a22cf5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/2903816340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cf56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.041 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 >> 172.21.15.145:0/2903816340 conn(0x556a22c600e0 msgr2=0x556a22cea6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f7184ff9700 0 client.26134 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:39.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 shutdown_connections 2023-08-25T04:55:39.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 1 -- 172.21.15.145:0/2903816340 wait complete. 2023-08-25T04:55:39.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.019+0000 7f71b73a0b80 20 client.26134 trim_cache size 0 max 16384 2023-08-25T04:55:39.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 Processor -- start 2023-08-25T04:55:39.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 -- start start 2023-08-25T04:55:39.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cf01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22abf920 con 0x556a22cf1500 2023-08-25T04:55:39.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22cbb1c0 con 0x556a22cf1e20 2023-08-25T04:55:39.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22ce1130 con 0x556a22cebc40 2023-08-25T04:55:39.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43942/0 (socket says 172.21.15.145:43942) 2023-08-25T04:55:39.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71a0f9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.038+0000 7f71a1f9c700 1 -- 172.21.15.145:0/3827201814 learned_addr learned my addr 172.21.15.145:0/3827201814 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:39.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cf01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 -- 172.21.15.145:0/3827201814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 msgr2=0x556a22cf01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cf01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 -- 172.21.15.145:0/3827201814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 msgr2=0x556a22cef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 -- 172.21.15.145:0/3827201814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a22a69de0 con 0x556a22cebc40 2023-08-25T04:55:39.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:55:39.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7194002230 tx=0x7f71940135b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6549cf620c76c878 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f7187fff700 1 -- 172.21.15.145:0/3827201814 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719400e040 con 0x556a22cebc40 2023-08-25T04:55:39.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f7187fff700 1 -- 172.21.15.145:0/3827201814 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719400f040 con 0x556a22cebc40 2023-08-25T04:55:39.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.039+0000 7f7187fff700 1 -- 172.21.15.145:0/3827201814 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194008ae0 con 0x556a22cebc40 2023-08-25T04:55:39.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3827201814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 msgr2=0x556a22ceb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7194002230 tx=0x7f71940135b0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3827201814 shutdown_connections 2023-08-25T04:55:39.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cf01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cebc40 0x556a22ceb700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3827201814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cf1500 0x556a22cef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3827201814 >> 172.21.15.145:0/3827201814 conn(0x556a22c600e0 msgr2=0x556a22ceef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3827201814 shutdown_connections 2023-08-25T04:55:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.040+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3827201814 wait complete. 2023-08-25T04:55:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 -- start start 2023-08-25T04:55:39.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 Processor -- start 2023-08-25T04:55:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cebc40 0x556a22d52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cfa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22cfae60 con 0x556a22cebc40 2023-08-25T04:55:39.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22cfafd0 con 0x556a22cf1e20 2023-08-25T04:55:39.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.041+0000 7f71b73a0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a22cfb190 con 0x556a22cf1500 2023-08-25T04:55:39.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71b73a0b80 10 client.0 _ll_register_callbacks cb 0x7f7194013b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:55:39.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a1f9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cebc40 0x556a22d52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:43958/0 (socket says 172.21.15.145:43958) 2023-08-25T04:55:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 -- 172.21.15.145:0/3614645495 learned_addr learned my addr 172.21.15.145:0/3614645495 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:55:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 -- 172.21.15.145:0/3614645495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 msgr2=0x556a22cfa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cfa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 -- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cebc40 msgr2=0x556a22d52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:55:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cebc40 0x556a22d52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 -- 172.21.15.145:0/3614645495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a22abd630 con 0x556a22cf1500 2023-08-25T04:55:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f71a0f9a700 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c00b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d4f8b78cd9865a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:55:39.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.042+0000 7f7184ff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:55:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c01d040 con 0x556a22cf1500 2023-08-25T04:55:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f718c00be70 con 0x556a22cf1500 2023-08-25T04:55:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a22cde8b0 con 0x556a22cf1500 2023-08-25T04:55:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718c005240 con 0x556a22cf1500 2023-08-25T04:55:39.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 10 client.26140 fetch_fsmap learned FSMap version 35 2023-08-25T04:55:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f719c10de80 con 0x556a22cf1500 2023-08-25T04:55:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a22cde8b0 con 0x556a22cf1500 2023-08-25T04:55:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f718c032ca0 con 0x556a22cf1500 2023-08-25T04:55:39.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:55:39.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:55:39.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 20 client.26140 populate_metadata read hostname 'smithi145' 2023-08-25T04:55:39.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d5a2d0 0x556a22d5c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:55:39.062 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 4 client.26140 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:55:39.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.043+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556a22c36970 con 0x556a22d5a2d0 2023-08-25T04:55:39.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d5a2d0 0x556a22d5c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:55:39.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f71a1f9c700 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d5a2d0 0x556a22d5c6c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cce2d7474dd7907 server_cookie=4f7bd7e5594830f8 in_seq=0 out_seq=0 2023-08-25T04:55:39.063 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f7184ff9700 10 client.26140 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f7184ff9700 1 -- 172.21.15.145:0/3614645495 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556a22c36970 con 0x556a22d5a2d0 2023-08-25T04:55:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f7184ff9700 10 client.26140 handle_command_reply: tid=0 2023-08-25T04:55:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.044+0000 7f71b73a0b80 1 client.26140 shutdown 2023-08-25T04:55:39.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 msgr2=0x556a22d52e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c00b2c0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 shutdown_connections 2023-08-25T04:55:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556a22cf1e20 0x556a22cfa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a22cebc40 0x556a22d52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556a22d5a2d0 0x556a22d5c6c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 --2- 172.21.15.145:0/3614645495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a22cf1500 0x556a22d52e40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 >> 172.21.15.145:0/3614645495 conn(0x556a22c600e0 msgr2=0x556a22ce60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f7184ff9700 0 client.26140 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:55:39.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 shutdown_connections 2023-08-25T04:55:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 1 -- 172.21.15.145:0/3614645495 wait complete. 2023-08-25T04:55:39.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.045+0000 7f71b73a0b80 20 client.26140 trim_cache size 0 max 16384 2023-08-25T04:55:39.067 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:55:39.068 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:55:39.068 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:55:39.068 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:55:39.068 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:55:39.069 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:55:39.069 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:55:39.069 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:55:39.069 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:55:39.069 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:55:39.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 msgr2=0x7f71880a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.071 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 0x7f71880a26a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71980c90a0 tx=0x7f7198009f60 comp rx=0 tx=0).stop 2023-08-25T04:55:39.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 msgr2=0x7f719c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:55:39.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f719800eb30 tx=0x7f719800ee40 comp rx=0 tx=0).stop 2023-08-25T04:55:39.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 shutdown_connections 2023-08-25T04:55:39.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f71880a01e0 0x7f71880a26a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.072 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f719c1066c0 0x7f719c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c1006b0 0x7f719c19e6a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 --2- 172.21.15.145:0/3950035318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f719c101070 0x7f719c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:55:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 >> 172.21.15.145:0/3950035318 conn(0x7f719c0fc160 msgr2=0x7f719c10bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:55:39.073 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 shutdown_connections 2023-08-25T04:55:39.074 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:55:39.046+0000 7f71a279d700 1 -- 172.21.15.145:0/3950035318 wait complete. 2023-08-25T04:55:39.074 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:55:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:39 smithi145 ceph-mon[97681]: pgmap v328: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:40.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:39 smithi162 ceph-mon[130941]: pgmap v328: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:40.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:39 smithi183 ceph-mon[131414]: pgmap v328: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:41 smithi145 ceph-mon[97681]: pgmap v329: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:41 smithi162 ceph-mon[130941]: pgmap v329: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:42.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:41 smithi183 ceph-mon[131414]: pgmap v329: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:43 smithi145 ceph-mon[97681]: pgmap v330: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:43 smithi162 ceph-mon[130941]: pgmap v330: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:44.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:43 smithi183 ceph-mon[131414]: pgmap v330: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:45 smithi145 ceph-mon[97681]: pgmap v331: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:45 smithi162 ceph-mon[130941]: pgmap v331: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:46.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:45 smithi183 ceph-mon[131414]: pgmap v331: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:48.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:47 smithi145 ceph-mon[97681]: pgmap v332: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:48.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:47 smithi162 ceph-mon[130941]: pgmap v332: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:47 smithi183 ceph-mon[131414]: pgmap v332: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:50.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:49 smithi145 ceph-mon[97681]: pgmap v333: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:50.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:49 smithi162 ceph-mon[130941]: pgmap v333: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:50.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:49 smithi183 ceph-mon[131414]: pgmap v333: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:52.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:51 smithi145 ceph-mon[97681]: pgmap v334: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:52.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:51 smithi162 ceph-mon[130941]: pgmap v334: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:52.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:51 smithi183 ceph-mon[131414]: pgmap v334: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:54.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:53 smithi145 ceph-mon[97681]: pgmap v335: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:54.734 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:53 smithi183 ceph-mon[131414]: pgmap v335: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:54.734 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:53 smithi162 ceph-mon[130941]: pgmap v335: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:55 smithi145 ceph-mon[97681]: pgmap v336: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:56.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:55 smithi162 ceph-mon[130941]: pgmap v336: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:56.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:55 smithi183 ceph-mon[131414]: pgmap v336: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:55:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:57 smithi145 ceph-mon[97681]: pgmap v337: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:57 smithi162 ceph-mon[130941]: pgmap v337: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:55:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:57 smithi183 ceph-mon[131414]: pgmap v337: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:00.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:55:59 smithi145 ceph-mon[97681]: pgmap v338: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:55:59 smithi162 ceph-mon[130941]: pgmap v338: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:55:59 smithi183 ceph-mon[131414]: pgmap v338: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:01 smithi145 ceph-mon[97681]: pgmap v339: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:02.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:01 smithi162 ceph-mon[130941]: pgmap v339: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:02.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:01 smithi183 ceph-mon[131414]: pgmap v339: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:03 smithi145 ceph-mon[97681]: pgmap v340: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:04.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:03 smithi162 ceph-mon[130941]: pgmap v340: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:03 smithi183 ceph-mon[131414]: pgmap v340: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:05 smithi145 ceph-mon[97681]: pgmap v341: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:06.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:05 smithi162 ceph-mon[130941]: pgmap v341: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:05 smithi183 ceph-mon[131414]: pgmap v341: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:07 smithi145 ceph-mon[97681]: pgmap v342: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:08.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:07 smithi162 ceph-mon[130941]: pgmap v342: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:07 smithi183 ceph-mon[131414]: pgmap v342: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:09.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:56:09.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 -- 172.21.15.145:0/1822374260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 msgr2=0x7f85ac106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 --2- 172.21.15.145:0/1822374260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac106c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f859c009990 tx=0x7f859c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 -- 172.21.15.145:0/1822374260 shutdown_connections 2023-08-25T04:56:09.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 --2- 172.21.15.145:0/1822374260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac101230 0x7f85ac10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 --2- 172.21.15.145:0/1822374260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac106c60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 --2- 172.21.15.145:0/1822374260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac100870 0x7f85ac100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 -- 172.21.15.145:0/1822374260 >> 172.21.15.145:0/1822374260 conn(0x7f85ac0fc3a0 msgr2=0x7f85ac0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 -- 172.21.15.145:0/1822374260 shutdown_connections 2023-08-25T04:56:09.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.307+0000 7f85b0dff700 1 -- 172.21.15.145:0/1822374260 wait complete. 2023-08-25T04:56:09.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.308+0000 7f85b0dff700 1 Processor -- start 2023-08-25T04:56:09.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.308+0000 7f85b0dff700 1 -- start start 2023-08-25T04:56:09.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac100870 0x7f85ac19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 0x7f85ac19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac199190 con 0x7f85ac101230 2023-08-25T04:56:09.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac199300 con 0x7f85ac100870 2023-08-25T04:56:09.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85b0dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1994c0 con 0x7f85ac106880 2023-08-25T04:56:09.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85aad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85aad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42914/0 (socket says 172.21.15.145:42914) 2023-08-25T04:56:09.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac100870 0x7f85ac19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85aad9d700 1 -- 172.21.15.145:0/3335375877 learned_addr learned my addr 172.21.15.145:0/3335375877 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.309+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 0x7f85ac19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 -- 172.21.15.145:0/3335375877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac100870 msgr2=0x7f85ac19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac100870 0x7f85ac19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 -- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 msgr2=0x7f85ac19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 0x7f85ac19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 -- 172.21.15.145:0/3335375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c0095f0 con 0x7f85ac106880 2023-08-25T04:56:09.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 0x7f85ac19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:09.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85aad9d700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85a4007ee0 tx=0x7f85a400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f859b7fe700 1 -- 172.21.15.145:0/3335375877 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400ddd0 con 0x7f85ac106880 2023-08-25T04:56:09.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f859b7fe700 1 -- 172.21.15.145:0/3335375877 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a400f040 con 0x7f85ac106880 2023-08-25T04:56:09.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac1997a0 con 0x7f85ac106880 2023-08-25T04:56:09.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f859b7fe700 1 -- 172.21.15.145:0/3335375877 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40048e0 con 0x7f85ac106880 2023-08-25T04:56:09.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.310+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac1a5960 con 0x7f85ac106880 2023-08-25T04:56:09.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.315+0000 7f859b7fe700 1 -- 172.21.15.145:0/3335375877 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f85a4003a00 con 0x7f85ac106880 2023-08-25T04:56:09.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 Processor -- start 2023-08-25T04:56:09.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 -- start start 2023-08-25T04:56:09.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b06356700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b063570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b062be7b0 con 0x561b06356c40 2023-08-25T04:56:09.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b062af780 con 0x561b0635cca0 2023-08-25T04:56:09.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.316+0000 7f85c46aab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b062ad0a0 con 0x561b0635c380 2023-08-25T04:56:09.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42916/0 (socket says 172.21.15.145:42916) 2023-08-25T04:56:09.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 -- 172.21.15.145:0/1406726894 learned_addr learned my addr 172.21.15.145:0/1406726894 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:09.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85aa59c700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b06356700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b063570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 -- 172.21.15.145:0/1406726894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 msgr2=0x561b06356700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b06356700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 -- 172.21.15.145:0/1406726894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 msgr2=0x561b063570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b063570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 -- 172.21.15.145:0/1406726894 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b06128630 con 0x561b0635c380 2023-08-25T04:56:09.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85aa59c700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b06356700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:56:09.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85a0012870 tx=0x7f85a0009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79308b02df6528c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.317+0000 7f859b7fe700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 0x7f85940a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f8598ff9700 1 -- 172.21.15.145:0/1406726894 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0012e90 con 0x561b0635c380 2023-08-25T04:56:09.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f8598ff9700 1 -- 172.21.15.145:0/1406726894 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a000f910 con 0x561b0635c380 2023-08-25T04:56:09.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f8598ff9700 1 -- 172.21.15.145:0/1406726894 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a001e860 con 0x561b0635c380 2023-08-25T04:56:09.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f859b7fe700 1 -- 172.21.15.145:0/3335375877 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f85a40cb250 con 0x7f85ac106880 2023-08-25T04:56:09.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f85aa59c700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 0x7f85940a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f85c46aab80 1 -- 172.21.15.145:0/1406726894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 msgr2=0x561b0635c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.318+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85a0012870 tx=0x7f85a0009790 comp rx=0 tx=0).stop 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 -- 172.21.15.145:0/1406726894 shutdown_connections 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b063570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635c380 0x561b0635c760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1406726894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b06356700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 -- 172.21.15.145:0/1406726894 >> 172.21.15.145:0/1406726894 conn(0x561b062cb0e0 msgr2=0x561b06351410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85aa59c700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 0x7f85940a2600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x561b0635d100 tx=0x7f859c005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 -- 172.21.15.145:0/1406726894 shutdown_connections 2023-08-25T04:56:09.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 -- 172.21.15.145:0/1406726894 wait complete. 2023-08-25T04:56:09.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.319+0000 7f85c46aab80 1 -- start start 2023-08-25T04:56:09.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 Processor -- start 2023-08-25T04:56:09.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b0635fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 0x561b06360610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b062a8990 con 0x561b0635cca0 2023-08-25T04:56:09.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b06246bb0 con 0x561b0635c380 2023-08-25T04:56:09.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b062468b0 con 0x561b06356c40 2023-08-25T04:56:09.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85c46aab80 10 client.0 _ll_register_callbacks cb 0x561b06190a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:56:09.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42932/0 (socket says 172.21.15.145:42932) 2023-08-25T04:56:09.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85aa59c700 1 -- 172.21.15.145:0/3108818493 learned_addr learned my addr 172.21.15.145:0/3108818493 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:09.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.320+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b0635fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aad9d700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 0x561b06360610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 -- 172.21.15.145:0/3108818493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 msgr2=0x561b0635fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b0635fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 -- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 msgr2=0x561b06360610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 0x561b06360610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 -- 172.21.15.145:0/3108818493 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac10e040 con 0x561b06356c40 2023-08-25T04:56:09.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aad9d700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 0x561b06360610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:56:09.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85aa59c700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f859c015000 tx=0x7f859c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4984f0b6c79c4a43 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f8591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:56:09.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00bed0 con 0x561b06356c40 2023-08-25T04:56:09.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b063261c0 con 0x561b06356c40 2023-08-25T04:56:09.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f859c026460 con 0x561b06356c40 2023-08-25T04:56:09.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.321+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f859c0376a0 con 0x561b06356c40 2023-08-25T04:56:09.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 10 client.26152 fetch_fsmap learned FSMap version 35 2023-08-25T04:56:09.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f85a001a040 con 0x561b06356c40 2023-08-25T04:56:09.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b063261c0 con 0x561b06356c40 2023-08-25T04:56:09.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f859c026620 con 0x561b06356c40 2023-08-25T04:56:09.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 10 client.26152 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:56:09.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 10 client.26152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:56:09.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 20 client.26152 populate_metadata read hostname 'smithi145' 2023-08-25T04:56:09.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b06370750 0x561b06372b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 4 client.26152 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:56:09.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.322+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b0624eba0 con 0x561b06370750 2023-08-25T04:56:09.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.323+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b06370750 0x561b06372b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.323+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b06370750 0x561b06372b70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613eed65cfad3bd5 server_cookie=b794fd87ced0aad0 in_seq=0 out_seq=0 2023-08-25T04:56:09.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.323+0000 7f8591ffb700 10 client.26152 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:09.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f8591ffb700 1 -- 172.21.15.145:0/3108818493 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561b0624eba0 con 0x561b06370750 2023-08-25T04:56:09.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f8591ffb700 10 client.26152 handle_command_reply: tid=0 2023-08-25T04:56:09.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 client.26152 shutdown 2023-08-25T04:56:09.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 msgr2=0x561b06362c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f859c015000 tx=0x7f859c009310 comp rx=0 tx=0).stop 2023-08-25T04:56:09.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 shutdown_connections 2023-08-25T04:56:09.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635cca0 0x561b06360610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b06370750 0x561b06372b70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b06362c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 --2- 172.21.15.145:0/3108818493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b0635fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f8591ffb700 0 client.26152 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:09.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 >> 172.21.15.145:0/3108818493 conn(0x561b062cb0e0 msgr2=0x561b0634bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 shutdown_connections 2023-08-25T04:56:09.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.324+0000 7f85c46aab80 1 -- 172.21.15.145:0/3108818493 wait complete. 2023-08-25T04:56:09.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.325+0000 7f85c46aab80 20 client.26152 trim_cache size 0 max 16384 2023-08-25T04:56:09.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 Processor -- start 2023-08-25T04:56:09.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 -- start start 2023-08-25T04:56:09.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635c380 0x561b063627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b0635a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0612a920 con 0x561b0635c380 2023-08-25T04:56:09.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0636de30 con 0x561b0635cca0 2023-08-25T04:56:09.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.351+0000 7f85c46aab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b0634dc80 con 0x561b06356c40 2023-08-25T04:56:09.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42934/0 (socket says 172.21.15.145:42934) 2023-08-25T04:56:09.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 -- 172.21.15.145:0/1680033767 learned_addr learned my addr 172.21.15.145:0/1680033767 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:09.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635c380 0x561b063627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aad9d700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b0635a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 -- 172.21.15.145:0/1680033767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 msgr2=0x561b0635a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b0635a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 -- 172.21.15.145:0/1680033767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635c380 msgr2=0x561b063627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635c380 0x561b063627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 -- 172.21.15.145:0/1680033767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b060d4de0 con 0x561b06356c40 2023-08-25T04:56:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aad9d700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b0635a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:09.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.352+0000 7f85aa59c700 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f859c009970 tx=0x7f859c00b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874531ac826df7a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.353+0000 7f85997fa700 1 -- 172.21.15.145:0/1680033767 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c026ba0 con 0x561b06356c40 2023-08-25T04:56:09.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.353+0000 7f85997fa700 1 -- 172.21.15.145:0/1680033767 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f859c01e040 con 0x561b06356c40 2023-08-25T04:56:09.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.353+0000 7f85997fa700 1 -- 172.21.15.145:0/1680033767 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00f8a0 con 0x561b06356c40 2023-08-25T04:56:09.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- 172.21.15.145:0/1680033767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 msgr2=0x561b0635f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f859c009970 tx=0x7f859c00b850 comp rx=0 tx=0).stop 2023-08-25T04:56:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- 172.21.15.145:0/1680033767 shutdown_connections 2023-08-25T04:56:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635cca0 0x561b0635a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b06356c40 0x561b0635f6e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 --2- 172.21.15.145:0/1680033767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b0635c380 0x561b063627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- 172.21.15.145:0/1680033767 >> 172.21.15.145:0/1680033767 conn(0x561b062cb0e0 msgr2=0x561b0636ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- 172.21.15.145:0/1680033767 shutdown_connections 2023-08-25T04:56:09.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- 172.21.15.145:0/1680033767 wait complete. 2023-08-25T04:56:09.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.354+0000 7f85c46aab80 1 -- start start 2023-08-25T04:56:09.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 Processor -- start 2023-08-25T04:56:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b06358fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b06359510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b06359b60 con 0x561b06356c40 2023-08-25T04:56:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b06358340 con 0x561b0635c380 2023-08-25T04:56:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b06358500 con 0x561b0635cca0 2023-08-25T04:56:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.355+0000 7f85c46aab80 10 client.0 _ll_register_callbacks cb 0x7f859c005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:56:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b06358fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.368 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:42948/0 (socket says 172.21.15.145:42948) 2023-08-25T04:56:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b06358fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41936/0 (socket says 172.21.15.145:41936) 2023-08-25T04:56:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 -- 172.21.15.145:0/26032837 learned_addr learned my addr 172.21.15.145:0/26032837 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b06359510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 -- 172.21.15.145:0/26032837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 msgr2=0x561b06359510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b06359510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 -- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 msgr2=0x561b06358fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b06358fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 -- 172.21.15.145:0/26032837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a001a040 con 0x561b0635cca0 2023-08-25T04:56:09.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85a9d9b700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b06359510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:09.373 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f85aad9d700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85a4098550 tx=0x7f85a40949c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c382d00c7449e226 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.356+0000 7f8591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:56:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4098cc0 con 0x561b0635cca0 2023-08-25T04:56:09.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f85a40985c0 con 0x561b0635cca0 2023-08-25T04:56:09.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b063498b0 con 0x561b0635cca0 2023-08-25T04:56:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a409c3f0 con 0x561b0635cca0 2023-08-25T04:56:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 10 client.26158 fetch_fsmap learned FSMap version 35 2023-08-25T04:56:09.377 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561b060bdc60 con 0x561b0635cca0 2023-08-25T04:56:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a40b6030 con 0x561b0635cca0 2023-08-25T04:56:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f85a40830b0 con 0x561b0635cca0 2023-08-25T04:56:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 10 client.26158 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:56:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:56:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 20 client.26158 populate_metadata read hostname 'smithi145' 2023-08-25T04:56:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b063c53b0 0x561b063c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 4 client.26158 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:56:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.357+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b062a1970 con 0x561b063c53b0 2023-08-25T04:56:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f85aa59c700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b063c53b0 0x561b063c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f85aa59c700 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b063c53b0 0x561b063c77d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21d12cb3d0909c0 server_cookie=817028bbcf460903 in_seq=0 out_seq=0 2023-08-25T04:56:09.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f8591ffb700 10 client.26158 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:09.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f8591ffb700 1 -- 172.21.15.145:0/26032837 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561b062a1970 con 0x561b063c53b0 2023-08-25T04:56:09.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f8591ffb700 10 client.26158 handle_command_reply: tid=0 2023-08-25T04:56:09.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.358+0000 7f85c46aab80 1 client.26158 shutdown 2023-08-25T04:56:09.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 msgr2=0x561b06357e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85a4098550 tx=0x7f85a40949c0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 shutdown_connections 2023-08-25T04:56:09.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x561b063c53b0 0x561b063c77d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b0635cca0 0x561b06357e00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561b06356c40 0x561b06358fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 --2- 172.21.15.145:0/26032837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561b0635c380 0x561b06359510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 >> 172.21.15.145:0/26032837 conn(0x561b062cb0e0 msgr2=0x561b0635a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f8591ffb700 0 client.26158 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 shutdown_connections 2023-08-25T04:56:09.383 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 1 -- 172.21.15.145:0/26032837 wait complete. 2023-08-25T04:56:09.383 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.359+0000 7f85c46aab80 20 client.26158 trim_cache size 0 max 16384 2023-08-25T04:56:09.383 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:56:09.384 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:56:09.384 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:56:09.384 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:56:09.384 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:56:09.385 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:56:09.385 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:56:09.385 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:56:09.385 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:56:09.385 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:56:09.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 msgr2=0x7f85940a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.387 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 0x7f85940a2600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x561b0635d100 tx=0x7f859c005fd0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 msgr2=0x7f85ac1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:09.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85a4007ee0 tx=0x7f85a400d3b0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 shutdown_connections 2023-08-25T04:56:09.388 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85ac106880 0x7f85ac1989c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f85940a0140 0x7f85940a2600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac100870 0x7f85ac19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.389 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 --2- 172.21.15.145:0/3335375877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85ac101230 0x7f85ac19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:09.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 >> 172.21.15.145:0/3335375877 conn(0x7f85ac0fc3a0 msgr2=0x7f85ac10bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:09.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 shutdown_connections 2023-08-25T04:56:09.390 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:09.360+0000 7f85b0dff700 1 -- 172.21.15.145:0/3335375877 wait complete. 2023-08-25T04:56:09.391 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:56:10.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:10 smithi145 ceph-mon[97681]: pgmap v343: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:56:10.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:10 smithi162 ceph-mon[130941]: pgmap v343: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:56:10.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:10 smithi183 ceph-mon[131414]: pgmap v343: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:56:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:12 smithi145 ceph-mon[97681]: pgmap v344: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:12 smithi145 ceph-mon[97681]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T04:56:12.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:12 smithi162 ceph-mon[130941]: pgmap v344: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:12 smithi162 ceph-mon[130941]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T04:56:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:12 smithi183 ceph-mon[131414]: pgmap v344: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:56:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:12 smithi183 ceph-mon[131414]: osdmap e99: 12 total, 12 up, 12 in 2023-08-25T04:56:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:14 smithi145 ceph-mon[97681]: pgmap v346: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:14.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:14 smithi162 ceph-mon[130941]: pgmap v346: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:14 smithi183 ceph-mon[131414]: pgmap v346: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:16 smithi145 ceph-mon[97681]: pgmap v347: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:16.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:16 smithi162 ceph-mon[130941]: pgmap v347: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:16.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:16 smithi183 ceph-mon[131414]: pgmap v347: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:56:18.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:18 smithi145 ceph-mon[97681]: pgmap v348: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:18.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:18 smithi162 ceph-mon[130941]: pgmap v348: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:18.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:18 smithi183 ceph-mon[131414]: pgmap v348: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:20 smithi145 ceph-mon[97681]: pgmap v349: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:20 smithi145 ceph-mon[97681]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T04:56:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:20 smithi162 ceph-mon[130941]: pgmap v349: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:20.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:20 smithi162 ceph-mon[130941]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T04:56:20.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:20 smithi183 ceph-mon[131414]: pgmap v349: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:20.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:20 smithi183 ceph-mon[131414]: osdmap e100: 12 total, 12 up, 12 in 2023-08-25T04:56:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:22 smithi145 ceph-mon[97681]: pgmap v351: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:56:22.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:22 smithi162 ceph-mon[130941]: pgmap v351: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:56:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:22 smithi183 ceph-mon[131414]: pgmap v351: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:56:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:24 smithi145 ceph-mon[97681]: pgmap v352: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:24.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:24 smithi162 ceph-mon[130941]: pgmap v352: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:24.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:24 smithi183 ceph-mon[131414]: pgmap v352: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:26 smithi145 ceph-mon[97681]: pgmap v353: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:26.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:26 smithi162 ceph-mon[130941]: pgmap v353: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:26.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:26 smithi183 ceph-mon[131414]: pgmap v353: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:56:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:28 smithi145 ceph-mon[97681]: pgmap v354: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:28.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:28 smithi162 ceph-mon[130941]: pgmap v354: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:28.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:28 smithi183 ceph-mon[131414]: pgmap v354: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:30 smithi145 ceph-mon[97681]: pgmap v355: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:30 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:56:30.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:30 smithi162 ceph-mon[130941]: pgmap v355: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:30.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:30 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:56:30.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:30 smithi183 ceph-mon[131414]: pgmap v355: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:56:30.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:30 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:56:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:56:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:56:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:56:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:56:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:56:31.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:56:31.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:56:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:56:31.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:56:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:32 smithi145 ceph-mon[97681]: pgmap v356: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:56:32.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:32 smithi162 ceph-mon[130941]: pgmap v356: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:56:32.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:32 smithi183 ceph-mon[131414]: pgmap v356: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:56:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:34 smithi145 ceph-mon[97681]: pgmap v357: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:34.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:34 smithi162 ceph-mon[130941]: pgmap v357: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:34.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:34 smithi183 ceph-mon[131414]: pgmap v357: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:36 smithi145 ceph-mon[97681]: pgmap v358: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:36.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:36 smithi162 ceph-mon[130941]: pgmap v358: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:36 smithi183 ceph-mon[131414]: pgmap v358: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:38.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:38 smithi145 ceph-mon[97681]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:38 smithi162 ceph-mon[130941]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:38 smithi183 ceph-mon[131414]: pgmap v359: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:39.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:56:39.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 -- 172.21.15.145:0/501900822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 msgr2=0x7f9acc0730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/501900822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc0730d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009990 tx=0x7f9ac8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 -- 172.21.15.145:0/501900822 shutdown_connections 2023-08-25T04:56:39.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/501900822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc10d140 0x7f9acc10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/501900822 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc0730d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/501900822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc073610 0x7f9acc10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 -- 172.21.15.145:0/501900822 >> 172.21.15.145:0/501900822 conn(0x7f9acc0fc3f0 msgr2=0x7f9acc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 -- 172.21.15.145:0/501900822 shutdown_connections 2023-08-25T04:56:39.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.625+0000 7f9ad37ff700 1 -- 172.21.15.145:0/501900822 wait complete. 2023-08-25T04:56:39.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.626+0000 7f9ad37ff700 1 Processor -- start 2023-08-25T04:56:39.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.626+0000 7f9ad37ff700 1 -- start start 2023-08-25T04:56:39.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc10d140 0x7f9acc19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19cba0 con 0x7f9acc072cf0 2023-08-25T04:56:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19cd10 con 0x7f9acc10d140 2023-08-25T04:56:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad37ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19ced0 con 0x7f9acc073610 2023-08-25T04:56:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad159b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40342/0 (socket says 172.21.15.145:40342) 2023-08-25T04:56:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/1634683459 learned_addr learned my addr 172.21.15.145:0/1634683459 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc10d140 0x7f9acc19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/1634683459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc10d140 msgr2=0x7f9acc19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc10d140 0x7f9acc19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 msgr2=0x7f9acc198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.627+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/1634683459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80095f0 con 0x7f9acc073610 2023-08-25T04:56:39.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9a0 tx=0x7f9abc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ac27fc700 1 -- 172.21.15.145:0/1634683459 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc009790 con 0x7f9acc073610 2023-08-25T04:56:39.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ac27fc700 1 -- 172.21.15.145:0/1634683459 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9abc00cbd0 con 0x7f9acc073610 2023-08-25T04:56:39.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc19d1b0 con 0x7f9acc073610 2023-08-25T04:56:39.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ac27fc700 1 -- 172.21.15.145:0/1634683459 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc018680 con 0x7f9acc073610 2023-08-25T04:56:39.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.628+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1a5b10 con 0x7f9acc073610 2023-08-25T04:56:39.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.631+0000 7f9ac27fc700 1 -- 172.21.15.145:0/1634683459 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f9abc00cd40 con 0x7f9acc073610 2023-08-25T04:56:39.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.633+0000 7f9ac27fc700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 0x7f9ab80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.633+0000 7f9ac27fc700 1 -- 172.21.15.145:0/1634683459 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f9abc014070 con 0x7f9acc073610 2023-08-25T04:56:39.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.633+0000 7f9ad159b700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 0x7f9ab80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 Processor -- start 2023-08-25T04:56:39.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 -- start start 2023-08-25T04:56:39.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63241760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.645 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 0x559b6323b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b631a37b0 con 0x559b63241380 2023-08-25T04:56:39.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b63194780 con 0x559b63241ca0 2023-08-25T04:56:39.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.634+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b631920a0 con 0x559b6323bc40 2023-08-25T04:56:39.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40356/0 (socket says 172.21.15.145:40356) 2023-08-25T04:56:39.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/2900388190 learned_addr learned my addr 172.21.15.145:0/2900388190 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:39.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad159b700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 0x7f9ab80a2600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b2e0 tx=0x7f9ac8005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad159b700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63241760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 0x559b6323b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/2900388190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 msgr2=0x559b6323b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 0x559b6323b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/2900388190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 msgr2=0x559b63241760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63241760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/2900388190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b6300d630 con 0x559b6323bc40 2023-08-25T04:56:39.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 0x559b6323b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:39.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad159b700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63241760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:39.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40093e0 tx=0x7f9ac4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d2a31b045a4cca server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.635+0000 7f9ab77fe700 1 -- 172.21.15.145:0/2900388190 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x559b6323bc40 2023-08-25T04:56:39.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ab77fe700 1 -- 172.21.15.145:0/2900388190 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac400f040 con 0x559b6323bc40 2023-08-25T04:56:39.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ab77fe700 1 -- 172.21.15.145:0/2900388190 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4014750 con 0x559b6323bc40 2023-08-25T04:56:39.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2900388190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 msgr2=0x559b6323c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40093e0 tx=0x7f9ac4009920 comp rx=0 tx=0).stop 2023-08-25T04:56:39.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2900388190 shutdown_connections 2023-08-25T04:56:39.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323c0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63241760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2900388190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241ca0 0x559b6323b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2900388190 >> 172.21.15.145:0/2900388190 conn(0x559b631b00e0 msgr2=0x559b63236410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.636+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2900388190 shutdown_connections 2023-08-25T04:56:39.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.637+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2900388190 wait complete. 2023-08-25T04:56:39.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.637+0000 7f9ae7049b80 1 -- start start 2023-08-25T04:56:39.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.637+0000 7f9ae7049b80 1 Processor -- start 2023-08-25T04:56:39.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b63247be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63244e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6318d990 con 0x559b63241380 2023-08-25T04:56:39.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6312bbb0 con 0x559b6323bc40 2023-08-25T04:56:39.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6312b8b0 con 0x559b63241ca0 2023-08-25T04:56:39.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ae7049b80 10 client.0 _ll_register_callbacks cb 0x559b63075a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:56:39.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ad1d9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ad1d9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40370/0 (socket says 172.21.15.145:40370) 2023-08-25T04:56:39.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/3283080050 learned_addr learned my addr 172.21.15.145:0/3283080050 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:39.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b63247be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.638+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63244e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/3283080050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 msgr2=0x559b63247be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b63247be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 msgr2=0x559b63244e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63244e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 -- 172.21.15.145:0/3283080050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4009090 con 0x559b63241ca0 2023-08-25T04:56:39.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b63247be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:56:39.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000f80 tx=0x7f9ac4015c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c8739cdf1d2791 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:56:39.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x559b63241ca0 2023-08-25T04:56:39.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b6320b1c0 con 0x559b63241ca0 2023-08-25T04:56:39.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f9ac401e680 con 0x559b63241ca0 2023-08-25T04:56:39.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.639+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ac401e840 con 0x559b63241ca0 2023-08-25T04:56:39.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 10 client.26170 fetch_fsmap learned FSMap version 35 2023-08-25T04:56:39.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559b62fb9de0 con 0x559b63241ca0 2023-08-25T04:56:39.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b6320b1c0 con 0x559b63241ca0 2023-08-25T04:56:39.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ac401db80 con 0x559b63241ca0 2023-08-25T04:56:39.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 10 client.26170 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:56:39.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:56:39.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 20 client.26170 populate_metadata read hostname 'smithi145' 2023-08-25T04:56:39.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b63260170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 4 client.26170 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:56:39.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b63133ba0 con 0x559b6325dd50 2023-08-25T04:56:39.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.640+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b63260170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.641+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b63260170 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb75b4486b4a03ef server_cookie=75c9b997c64a2f01 in_seq=0 out_seq=0 2023-08-25T04:56:39.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.641+0000 7f9ab4ff9700 10 client.26170 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:39.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/3283080050 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559b63133ba0 con 0x559b6325dd50 2023-08-25T04:56:39.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ab4ff9700 10 client.26170 handle_command_reply: tid=0 2023-08-25T04:56:39.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 client.26170 shutdown 2023-08-25T04:56:39.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 msgr2=0x559b632454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000f80 tx=0x7f9ac4015c20 comp rx=0 tx=0).stop 2023-08-25T04:56:39.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 shutdown_connections 2023-08-25T04:56:39.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b63260170 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241ca0 0x559b632454e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ab4ff9700 0 client.26170 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:39.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b63247be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3283080050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241380 0x559b63244e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 >> 172.21.15.145:0/3283080050 conn(0x559b631b00e0 msgr2=0x559b63230b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 shutdown_connections 2023-08-25T04:56:39.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.642+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3283080050 wait complete. 2023-08-25T04:56:39.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.643+0000 7f9ae7049b80 20 client.26170 trim_cache size 0 max 16384 2023-08-25T04:56:39.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 Processor -- start 2023-08-25T04:56:39.690 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 -- start start 2023-08-25T04:56:39.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241380 0x559b6323eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6300f920 con 0x559b63241ca0 2023-08-25T04:56:39.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b63232a60 con 0x559b63241380 2023-08-25T04:56:39.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b63248320 con 0x559b6323bc40 2023-08-25T04:56:39.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ad159b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.666+0000 7f9ad159b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40376/0 (socket says 172.21.15.145:40376) 2023-08-25T04:56:39.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 -- 172.21.15.145:0/3137742378 learned_addr learned my addr 172.21.15.145:0/3137742378 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:39.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241380 0x559b6323eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 -- 172.21.15.145:0/3137742378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241380 msgr2=0x559b6323eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241380 0x559b6323eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 -- 172.21.15.145:0/3137742378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 msgr2=0x559b6323f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 -- 172.21.15.145:0/3137742378 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b62fa2c60 con 0x559b6323bc40 2023-08-25T04:56:39.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ad159b700 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b6c0 tx=0x7f9ac8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd098bf12778ba9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ab7fff700 1 -- 172.21.15.145:0/3137742378 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80056c0 con 0x559b6323bc40 2023-08-25T04:56:39.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ab7fff700 1 -- 172.21.15.145:0/3137742378 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac802a030 con 0x559b6323bc40 2023-08-25T04:56:39.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.667+0000 7f9ab7fff700 1 -- 172.21.15.145:0/3137742378 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004c70 con 0x559b6323bc40 2023-08-25T04:56:39.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3137742378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 msgr2=0x559b6323b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b6c0 tx=0x7f9ac8020000 comp rx=0 tx=0).stop 2023-08-25T04:56:39.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3137742378 shutdown_connections 2023-08-25T04:56:39.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b6323bc40 0x559b6323b700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/3137742378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b63241380 0x559b6323eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3137742378 >> 172.21.15.145:0/3137742378 conn(0x559b631b00e0 msgr2=0x559b632360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.704 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3137742378 shutdown_connections 2023-08-25T04:56:39.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.668+0000 7f9ae7049b80 1 -- 172.21.15.145:0/3137742378 wait complete. 2023-08-25T04:56:39.705 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 -- start start 2023-08-25T04:56:39.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 Processor -- start 2023-08-25T04:56:39.706 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b6323e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.707 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6323ea50 con 0x559b63241ca0 2023-08-25T04:56:39.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6323d880 con 0x559b6323bc40 2023-08-25T04:56:39.708 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6323da40 con 0x559b63241380 2023-08-25T04:56:39.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.669+0000 7f9ae7049b80 10 client.0 _ll_register_callbacks cb 0x7f9ac40092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:56:39.709 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:40380/0 (socket says 172.21.15.145:40380) 2023-08-25T04:56:39.710 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/2666201876 learned_addr learned my addr 172.21.15.145:0/2666201876 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:56:39.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad159b700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b6323e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.711 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad1d9c700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/2666201876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 msgr2=0x559b6323e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.712 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b6323e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 msgr2=0x559b6323d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:56:39.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.713 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 -- 172.21.15.145:0/2666201876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc110280 con 0x559b63241380 2023-08-25T04:56:39.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ad0d9a700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009230 tx=0x7f9abc0c9530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8f13438e38701d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:56:39.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:56:39.714 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc098f90 con 0x559b63241380 2023-08-25T04:56:39.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.670+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f9abc09cdc0 con 0x559b63241380 2023-08-25T04:56:39.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b6322e8b0 con 0x559b63241380 2023-08-25T04:56:39.715 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9abc0a0da0 con 0x559b63241380 2023-08-25T04:56:39.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 10 client.26176 fetch_fsmap learned FSMap version 35 2023-08-25T04:56:39.716 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559b62fe0250 con 0x559b63241380 2023-08-25T04:56:39.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b6322e8b0 con 0x559b63241380 2023-08-25T04:56:39.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9abc0b80b0 con 0x559b63241380 2023-08-25T04:56:39.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 10 client.26176 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:56:39.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:56:39.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 20 client.26176 populate_metadata read hostname 'smithi145' 2023-08-25T04:56:39.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b632b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:56:39.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 4 client.26176 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:56:39.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b63186970 con 0x559b6325dd50 2023-08-25T04:56:39.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.671+0000 7f9ad159b700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b632b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:56:39.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ab4ff9700 10 client.26176 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:39.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ad159b700 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b632b4d20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5170188ce6f899e4 server_cookie=cbb75f5cb8d147b0 in_seq=0 out_seq=0 2023-08-25T04:56:39.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ab4ff9700 1 -- 172.21.15.145:0/2666201876 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559b63186970 con 0x559b6325dd50 2023-08-25T04:56:39.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ab4ff9700 10 client.26176 handle_command_reply: tid=0 2023-08-25T04:56:39.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ae7049b80 1 client.26176 shutdown 2023-08-25T04:56:39.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 msgr2=0x559b6323ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.672+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009230 tx=0x7f9abc0c9530 comp rx=0 tx=0).stop 2023-08-25T04:56:39.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 shutdown_connections 2023-08-25T04:56:39.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559b63241ca0 0x559b6323d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b6323bc40 0x559b6323e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x559b6325dd50 0x559b632b4d20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 --2- 172.21.15.145:0/2666201876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b63241380 0x559b6323ce00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 >> 172.21.15.145:0/2666201876 conn(0x559b631b00e0 msgr2=0x559b632302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ab4ff9700 0 client.26176 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:56:39.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 shutdown_connections 2023-08-25T04:56:39.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 1 -- 172.21.15.145:0/2666201876 wait complete. 2023-08-25T04:56:39.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ae7049b80 20 client.26176 trim_cache size 0 max 16384 2023-08-25T04:56:39.728 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:56:39.729 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:56:39.729 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:56:39.729 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:56:39.730 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:56:39.730 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:56:39.731 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:56:39.731 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:56:39.732 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:56:39.732 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:56:39.733 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 msgr2=0x7f9ab80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 0x7f9ab80a2600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b2e0 tx=0x7f9ac8005fd0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 msgr2=0x7f9acc1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:56:39.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.673+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9a0 tx=0x7f9abc00ecb0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.734 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 shutdown_connections 2023-08-25T04:56:39.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9ab80a0140 0x7f9ab80a2600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9acc10d140 0x7f9acc19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.735 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9acc072cf0 0x7f9acc198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 --2- 172.21.15.145:0/1634683459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9acc073610 0x7f9acc1991f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:56:39.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 >> 172.21.15.145:0/1634683459 conn(0x7f9acc0fc3f0 msgr2=0x7f9acc10f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:56:39.736 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 shutdown_connections 2023-08-25T04:56:39.737 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:56:39.674+0000 7f9ad37ff700 1 -- 172.21.15.145:0/1634683459 wait complete. 2023-08-25T04:56:39.738 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:56:40.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:40 smithi145 ceph-mon[97681]: pgmap v360: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:40.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:40 smithi162 ceph-mon[130941]: pgmap v360: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:40.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:40 smithi183 ceph-mon[131414]: pgmap v360: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:42 smithi145 ceph-mon[97681]: pgmap v361: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:42 smithi162 ceph-mon[130941]: pgmap v361: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:42.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:42 smithi183 ceph-mon[131414]: pgmap v361: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:44 smithi145 ceph-mon[97681]: pgmap v362: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:44.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:44 smithi162 ceph-mon[130941]: pgmap v362: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:44.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:44 smithi183 ceph-mon[131414]: pgmap v362: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:46 smithi145 ceph-mon[97681]: pgmap v363: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:46 smithi162 ceph-mon[130941]: pgmap v363: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:46.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:46 smithi183 ceph-mon[131414]: pgmap v363: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:48.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:48 smithi145 ceph-mon[97681]: pgmap v364: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:48.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:48 smithi162 ceph-mon[130941]: pgmap v364: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:48 smithi183 ceph-mon[131414]: pgmap v364: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:49.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:49 smithi162 ceph-mon[130941]: pgmap v365: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:49.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:49 smithi183 ceph-mon[131414]: pgmap v365: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:49.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:49 smithi145 ceph-mon[97681]: pgmap v365: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:52.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:51 smithi145 ceph-mon[97681]: pgmap v366: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:51 smithi162 ceph-mon[130941]: pgmap v366: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:51 smithi183 ceph-mon[131414]: pgmap v366: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:53 smithi145 ceph-mon[97681]: pgmap v367: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:54.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:53 smithi162 ceph-mon[130941]: pgmap v367: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:53 smithi183 ceph-mon[131414]: pgmap v367: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:56.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:55 smithi145 ceph-mon[97681]: pgmap v368: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:55 smithi162 ceph-mon[130941]: pgmap v368: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:55 smithi183 ceph-mon[131414]: pgmap v368: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:56:58.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:57 smithi145 ceph-mon[97681]: pgmap v369: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:58.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:57 smithi162 ceph-mon[130941]: pgmap v369: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:56:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:57 smithi183 ceph-mon[131414]: pgmap v369: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:00.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:56:59 smithi145 ceph-mon[97681]: pgmap v370: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:00.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:56:59 smithi162 ceph-mon[130941]: pgmap v370: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:56:59 smithi183 ceph-mon[131414]: pgmap v370: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:02.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:01 smithi145 ceph-mon[97681]: pgmap v371: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:02.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:01 smithi162 ceph-mon[130941]: pgmap v371: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:01 smithi183 ceph-mon[131414]: pgmap v371: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:04.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:03 smithi145 ceph-mon[97681]: pgmap v372: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:03 smithi162 ceph-mon[130941]: pgmap v372: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:03 smithi183 ceph-mon[131414]: pgmap v372: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:06.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:05 smithi145 ceph-mon[97681]: pgmap v373: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:05 smithi162 ceph-mon[130941]: pgmap v373: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:06.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:05 smithi183 ceph-mon[131414]: pgmap v373: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:08.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:07 smithi145 ceph-mon[97681]: pgmap v374: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:07 smithi162 ceph-mon[130941]: pgmap v374: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:07 smithi183 ceph-mon[131414]: pgmap v374: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:09.739 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:57:09.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 -- 172.21.15.145:0/3674862330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda01068a0 msgr2=0x7feda0106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:09.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 --2- 172.21.15.145:0/3674862330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda01068a0 0x7feda0106c80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009990 tx=0x7fed9c009ca0 comp rx=0 tx=0).stop 2023-08-25T04:57:09.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 -- 172.21.15.145:0/3674862330 shutdown_connections 2023-08-25T04:57:09.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 --2- 172.21.15.145:0/3674862330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0101250 0x7feda010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:09.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 --2- 172.21.15.145:0/3674862330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda01068a0 0x7feda0106c80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:09.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 --2- 172.21.15.145:0/3674862330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:09.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.990+0000 7feda83ff700 1 -- 172.21.15.145:0/3674862330 >> 172.21.15.145:0/3674862330 conn(0x7feda00fc380 msgr2=0x7feda00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:09.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.991+0000 7feda83ff700 1 -- 172.21.15.145:0/3674862330 shutdown_connections 2023-08-25T04:57:09.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.991+0000 7feda83ff700 1 -- 172.21.15.145:0/3674862330 wait complete. 2023-08-25T04:57:09.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.992+0000 7feda83ff700 1 Processor -- start 2023-08-25T04:57:09.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.992+0000 7feda83ff700 1 -- start start 2023-08-25T04:57:09.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.992+0000 7feda83ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:09.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda83ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda0101250 0x7feda0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:09.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda83ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 0x7feda00736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda83ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda01a4e50 con 0x7feda0101250 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda83ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda01a4fc0 con 0x7feda0100890 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda83ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda01a5180 con 0x7feda01068a0 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:47318/0 (socket says 172.21.15.145:47318) 2023-08-25T04:57:09.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 -- 172.21.15.145:0/3897203189 learned_addr learned my addr 172.21.15.145:0/3897203189 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:09.999 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda599a700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda0101250 0x7feda0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:09.999 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda699c700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 0x7feda00736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:09.999 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 -- 172.21.15.145:0/3897203189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 msgr2=0x7feda00736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 0x7feda00736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 -- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda0101250 msgr2=0x7feda0073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda0101250 0x7feda0073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 -- 172.21.15.145:0/3897203189 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98009c40 con 0x7feda0100890 2023-08-25T04:57:10.000 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda699c700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 0x7feda00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:57:10.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.993+0000 7feda619b700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed9c000c00 tx=0x7fed9c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.994+0000 7fed977fe700 1 -- 172.21.15.145:0/3897203189 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c01d070 con 0x7feda0100890 2023-08-25T04:57:10.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.994+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c0095f0 con 0x7feda0100890 2023-08-25T04:57:10.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.994+0000 7fed977fe700 1 -- 172.21.15.145:0/3897203189 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c022470 con 0x7feda0100890 2023-08-25T04:57:10.001 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.994+0000 7fed977fe700 1 -- 172.21.15.145:0/3897203189 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c017650 con 0x7feda0100890 2023-08-25T04:57:10.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.994+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda01a57a0 con 0x7feda0100890 2023-08-25T04:57:10.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.997+0000 7fed977fe700 1 -- 172.21.15.145:0/3897203189 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fed9c0225e0 con 0x7feda0100890 2023-08-25T04:57:10.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.999+0000 7fed977fe700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 0x7fed8c0a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:09.999+0000 7fed977fe700 1 -- 172.21.15.145:0/3897203189 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fed9c0ccdc0 con 0x7feda0100890 2023-08-25T04:57:10.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.000+0000 7feda599a700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 0x7fed8c0a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.000+0000 7fedbbd2fb80 1 Processor -- start 2023-08-25T04:57:10.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.000+0000 7fedbbd2fb80 1 -- start start 2023-08-25T04:57:10.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.000+0000 7feda599a700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 0x7fed8c0a2600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feda00743c0 tx=0x7fed90009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.000+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b53b90 0x55a208b53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208aad7b0 con 0x55a208b4e450 2023-08-25T04:57:10.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208a9e780 con 0x55a208b53b90 2023-08-25T04:57:10.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208a9c0a0 con 0x55a208b544b0 2023-08-25T04:57:10.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55840/0 (socket says 172.21.15.145:55840) 2023-08-25T04:57:10.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 -- 172.21.15.145:0/184812381 learned_addr learned my addr 172.21.15.145:0/184812381 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:10.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda619b700 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b53b90 0x55a208b53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda699c700 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 -- 172.21.15.145:0/184812381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b53b90 msgr2=0x55a208b53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b53b90 0x55a208b53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 -- 172.21.15.145:0/184812381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 msgr2=0x55a208b4e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b4e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.001+0000 7feda599a700 1 -- 172.21.15.145:0/184812381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a208917630 con 0x55a208b544b0 2023-08-25T04:57:10.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.002+0000 7feda599a700 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed90011d50 tx=0x7fed90014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb25c460ec6b3432 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.002+0000 7fed94bfe700 1 -- 172.21.15.145:0/184812381 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000eb20 con 0x55a208b544b0 2023-08-25T04:57:10.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.002+0000 7fed94bfe700 1 -- 172.21.15.145:0/184812381 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed90021070 con 0x55a208b544b0 2023-08-25T04:57:10.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.002+0000 7fed94bfe700 1 -- 172.21.15.145:0/184812381 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9001d670 con 0x55a208b544b0 2023-08-25T04:57:10.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/184812381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 msgr2=0x55a208b4df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed90011d50 tx=0x7fed90014000 comp rx=0 tx=0).stop 2023-08-25T04:57:10.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/184812381 shutdown_connections 2023-08-25T04:57:10.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b4e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b53b90 0x55a208b53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/184812381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b4df10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/184812381 >> 172.21.15.145:0/184812381 conn(0x55a208aba0e0 msgr2=0x55a208b40410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:10.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/184812381 shutdown_connections 2023-08-25T04:57:10.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.003+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/184812381 wait complete. 2023-08-25T04:57:10.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.004+0000 7fedbbd2fb80 1 -- start start 2023-08-25T04:57:10.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.004+0000 7fedbbd2fb80 1 Processor -- start 2023-08-25T04:57:10.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 0x55a208b58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208a97990 con 0x55a208b4e450 2023-08-25T04:57:10.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208a35bb0 con 0x55a208b544b0 2023-08-25T04:57:10.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208a358b0 con 0x55a208b53b90 2023-08-25T04:57:10.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7fedbbd2fb80 10 client.0 _ll_register_callbacks cb 0x55a20897fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:57:10.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7feda619b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7feda619b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49222/0 (socket says 172.21.15.145:49222) 2023-08-25T04:57:10.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7feda619b700 1 -- 172.21.15.145:0/338237923 learned_addr learned my addr 172.21.15.145:0/338237923 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:10.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7feda599a700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.005+0000 7feda699c700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 0x55a208b58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 -- 172.21.15.145:0/338237923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 msgr2=0x55a208b57cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b57cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 -- 172.21.15.145:0/338237923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 msgr2=0x55a208b58200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 0x55a208b58200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 -- 172.21.15.145:0/338237923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed90017070 con 0x55a208b4e450 2023-08-25T04:57:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda699c700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 0x55a208b58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:57:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7feda619b700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed9c099ec0 tx=0x7fed9c099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6a81a80d232265 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fed89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:57:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c09dff0 con 0x55a208b4e450 2023-08-25T04:57:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fed9c0a1f40 con 0x55a208b4e450 2023-08-25T04:57:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a208b151c0 con 0x55a208b4e450 2023-08-25T04:57:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c09a630 con 0x55a208b4e450 2023-08-25T04:57:10.022 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fedbbd2fb80 10 client.16155 fetch_fsmap learned FSMap version 35 2023-08-25T04:57:10.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.006+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a2088c3de0 con 0x55a208b4e450 2023-08-25T04:57:10.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a208b151c0 con 0x55a208b4e450 2023-08-25T04:57:10.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed9c0a1ab0 con 0x55a208b4e450 2023-08-25T04:57:10.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:57:10.023 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:57:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 20 client.16155 populate_metadata read hostname 'smithi145' 2023-08-25T04:57:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208b67c90 0x55a208b6a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 4 client.16155 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:57:10.024 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.007+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a208a3dba0 con 0x55a208b67c90 2023-08-25T04:57:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.008+0000 7feda599a700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208b67c90 0x55a208b6a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.008+0000 7feda599a700 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208b67c90 0x55a208b6a080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e17b980f6ca5b76 server_cookie=3b25b67065430dfd in_seq=0 out_seq=0 2023-08-25T04:57:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.008+0000 7fed89ffb700 10 client.16155 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:10.025 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fed89ffb700 1 -- 172.21.15.145:0/338237923 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a208a3dba0 con 0x55a208b67c90 2023-08-25T04:57:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fed89ffb700 10 client.16155 handle_command_reply: tid=0 2023-08-25T04:57:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 client.16155 shutdown 2023-08-25T04:57:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 msgr2=0x55a208b57780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed9c099ec0 tx=0x7fed9c099ba0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.026 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 shutdown_connections 2023-08-25T04:57:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b544b0 0x55a208b58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b4e450 0x55a208b57780 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208b67c90 0x55a208b6a080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.027 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/338237923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fed89ffb700 0 client.16155 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 >> 172.21.15.145:0/338237923 conn(0x55a208aba0e0 msgr2=0x55a208b3ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 shutdown_connections 2023-08-25T04:57:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.009+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/338237923 wait complete. 2023-08-25T04:57:10.028 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.010+0000 7fedbbd2fb80 20 client.16155 trim_cache size 0 max 16384 2023-08-25T04:57:10.038 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 Processor -- start 2023-08-25T04:57:10.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 -- start start 2023-08-25T04:57:10.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b4df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.039 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b544b0 0x55a208b57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208919920 con 0x55a208b544b0 2023-08-25T04:57:10.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208b41150 con 0x55a208b4e450 2023-08-25T04:57:10.040 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208b3cc80 con 0x55a208b53b90 2023-08-25T04:57:10.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7feda599a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7feda599a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55864/0 (socket says 172.21.15.145:55864) 2023-08-25T04:57:10.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7feda599a700 1 -- 172.21.15.145:0/544090556 learned_addr learned my addr 172.21.15.145:0/544090556 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:10.042 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7feda699c700 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b544b0 0x55a208b57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.037+0000 7feda619b700 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b4df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 -- 172.21.15.145:0/544090556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 msgr2=0x55a208b4df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b4df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.043 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 -- 172.21.15.145:0/544090556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b544b0 msgr2=0x55a208b57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b544b0 0x55a208b57cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 -- 172.21.15.145:0/544090556 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2088acc60 con 0x55a208b53b90 2023-08-25T04:57:10.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7feda599a700 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fed90011d50 tx=0x7fed9001cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b2ff996689ba79 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7fed953ff700 1 -- 172.21.15.145:0/544090556 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000eb20 con 0x55a208b53b90 2023-08-25T04:57:10.044 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7fed953ff700 1 -- 172.21.15.145:0/544090556 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed90021070 con 0x55a208b53b90 2023-08-25T04:57:10.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.038+0000 7fed953ff700 1 -- 172.21.15.145:0/544090556 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9001c410 con 0x55a208b53b90 2023-08-25T04:57:10.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/544090556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 msgr2=0x55a208b5efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fed90011d50 tx=0x7fed9001cbf0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/544090556 shutdown_connections 2023-08-25T04:57:10.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b544b0 0x55a208b57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b4df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/544090556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b53b90 0x55a208b5efb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/544090556 >> 172.21.15.145:0/544090556 conn(0x55a208aba0e0 msgr2=0x55a208b518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:10.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/544090556 shutdown_connections 2023-08-25T04:57:10.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.039+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/544090556 wait complete. 2023-08-25T04:57:10.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.040+0000 7fedbbd2fb80 1 -- start start 2023-08-25T04:57:10.047 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.040+0000 7fedbbd2fb80 1 Processor -- start 2023-08-25T04:57:10.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b5e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.050 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b53b90 0x55a208b5ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208b5d6a0 con 0x55a208b53b90 2023-08-25T04:57:10.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208b5d810 con 0x55a208b4e450 2023-08-25T04:57:10.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a208b5d9d0 con 0x55a208b544b0 2023-08-25T04:57:10.051 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7fedbbd2fb80 10 client.0 _ll_register_callbacks cb 0x7fed9c099d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:57:10.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda699c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b5e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda699c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:55870/0 (socket says 172.21.15.145:55870) 2023-08-25T04:57:10.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda699c700 1 -- 172.21.15.145:0/2344253629 learned_addr learned my addr 172.21.15.145:0/2344253629 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:10.052 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda599a700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b53b90 0x55a208b5ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 -- 172.21.15.145:0/2344253629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 msgr2=0x55a208b5cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 -- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b53b90 msgr2=0x55a208b5ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:10.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b53b90 0x55a208b5ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda619b700 1 -- 172.21.15.145:0/2344253629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9002a040 con 0x55a208b4e450 2023-08-25T04:57:10.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.041+0000 7feda699c700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:57:10.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7feda619b700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b5e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fed9c096c30 tx=0x7fed9c005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e84a47a12a5ef0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:10.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:57:10.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0177b0 con 0x55a208b4e450 2023-08-25T04:57:10.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fed9c0a1f40 con 0x55a208b4e450 2023-08-25T04:57:10.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a208b388b0 con 0x55a208b4e450 2023-08-25T04:57:10.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c09dff0 con 0x55a208b4e450 2023-08-25T04:57:10.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fedbbd2fb80 10 client.26153 fetch_fsmap learned FSMap version 35 2023-08-25T04:57:10.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a2088ea250 con 0x55a208b4e450 2023-08-25T04:57:10.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a208b388b0 con 0x55a208b4e450 2023-08-25T04:57:10.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.042+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed9c07c050 con 0x55a208b4e450 2023-08-25T04:57:10.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 10 client.26153 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:57:10.056 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:57:10.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 20 client.26153 populate_metadata read hostname 'smithi145' 2023-08-25T04:57:10.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208bbc600 0x55a208bbe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:10.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 4 client.26153 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:57:10.057 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a208a90970 con 0x55a208bbc600 2023-08-25T04:57:10.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7feda599a700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208bbc600 0x55a208bbe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:10.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7feda599a700 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208bbc600 0x55a208bbe9f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eda383c95f2acb server_cookie=a15f90cf8cbcbd33 in_seq=0 out_seq=0 2023-08-25T04:57:10.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.043+0000 7fed89ffb700 10 client.26153 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:10.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fed89ffb700 1 -- 172.21.15.145:0/2344253629 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a208a90970 con 0x55a208bbc600 2023-08-25T04:57:10.058 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fed89ffb700 10 client.26153 handle_command_reply: tid=0 2023-08-25T04:57:10.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 client.26153 shutdown 2023-08-25T04:57:10.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 msgr2=0x55a208b5e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b5e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fed9c096c30 tx=0x7fed9c005ab0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 shutdown_connections 2023-08-25T04:57:10.059 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55a208bbc600 0x55a208bbe9f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a208b544b0 0x55a208b5cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a208b4e450 0x55a208b5e6e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fed89ffb700 0 client.26153 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:10.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 --2- 172.21.15.145:0/2344253629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a208b53b90 0x55a208b5ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.060 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 >> 172.21.15.145:0/2344253629 conn(0x55a208aba0e0 msgr2=0x55a208b57cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:10.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 shutdown_connections 2023-08-25T04:57:10.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 1 -- 172.21.15.145:0/2344253629 wait complete. 2023-08-25T04:57:10.061 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.044+0000 7fedbbd2fb80 20 client.26153 trim_cache size 0 max 16384 2023-08-25T04:57:10.062 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:57:10.062 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:57:10.062 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:57:10.062 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:57:10.062 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:57:10.063 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:57:10.063 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:57:10.063 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:57:10.063 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:57:10.063 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:57:10.064 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 msgr2=0x7fed8c0a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 0x7fed8c0a2600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feda00743c0 tx=0x7fed90009260 comp rx=0 tx=0).stop 2023-08-25T04:57:10.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 msgr2=0x7feda0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:10.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed9c000c00 tx=0x7fed9c005740 comp rx=0 tx=0).stop 2023-08-25T04:57:10.065 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 shutdown_connections 2023-08-25T04:57:10.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01068a0 0x7feda00736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fed8c0a0140 0x7fed8c0a2600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feda0100890 0x7feda0072c20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.066 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 --2- 172.21.15.145:0/3897203189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feda0101250 0x7feda0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:10.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 >> 172.21.15.145:0/3897203189 conn(0x7feda00fc380 msgr2=0x7feda00fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:10.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 shutdown_connections 2023-08-25T04:57:10.067 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:10.045+0000 7feda83ff700 1 -- 172.21.15.145:0/3897203189 wait complete. 2023-08-25T04:57:10.068 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:09 smithi145 ceph-mon[97681]: pgmap v375: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:57:10.069 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:57:10.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:09 smithi162 ceph-mon[130941]: pgmap v375: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:57:10.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:09 smithi183 ceph-mon[131414]: pgmap v375: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:57:12.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:11 smithi145 ceph-mon[97681]: pgmap v376: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:12.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:11 smithi145 ceph-mon[97681]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T04:57:12.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:11 smithi162 ceph-mon[130941]: pgmap v376: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:12.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:11 smithi162 ceph-mon[130941]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T04:57:12.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:11 smithi183 ceph-mon[131414]: pgmap v376: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:57:12.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:11 smithi183 ceph-mon[131414]: osdmap e101: 12 total, 12 up, 12 in 2023-08-25T04:57:14.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:13 smithi145 ceph-mon[97681]: pgmap v378: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:14.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:13 smithi162 ceph-mon[130941]: pgmap v378: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:13 smithi183 ceph-mon[131414]: pgmap v378: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:16.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:15 smithi145 ceph-mon[97681]: pgmap v379: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:16.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:15 smithi162 ceph-mon[130941]: pgmap v379: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:15 smithi183 ceph-mon[131414]: pgmap v379: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:57:18.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:17 smithi145 ceph-mon[97681]: pgmap v380: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:18.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:17 smithi162 ceph-mon[130941]: pgmap v380: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:17 smithi183 ceph-mon[131414]: pgmap v380: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:20.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:19 smithi145 ceph-mon[97681]: pgmap v381: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:20.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:19 smithi145 ceph-mon[97681]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T04:57:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:19 smithi162 ceph-mon[130941]: pgmap v381: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:19 smithi162 ceph-mon[130941]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T04:57:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:19 smithi183 ceph-mon[131414]: pgmap v381: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:19 smithi183 ceph-mon[131414]: osdmap e102: 12 total, 12 up, 12 in 2023-08-25T04:57:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:21 smithi162 ceph-mon[130941]: pgmap v383: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2023-08-25T04:57:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:21 smithi183 ceph-mon[131414]: pgmap v383: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2023-08-25T04:57:22.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:21 smithi145 ceph-mon[97681]: pgmap v383: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2023-08-25T04:57:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:23 smithi162 ceph-mon[130941]: pgmap v384: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:23 smithi183 ceph-mon[131414]: pgmap v384: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:23 smithi145 ceph-mon[97681]: pgmap v384: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:25 smithi162 ceph-mon[130941]: pgmap v385: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:25 smithi183 ceph-mon[131414]: pgmap v385: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:25 smithi145 ceph-mon[97681]: pgmap v385: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:57:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:27 smithi162 ceph-mon[130941]: pgmap v386: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:27 smithi183 ceph-mon[131414]: pgmap v386: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:27 smithi145 ceph-mon[97681]: pgmap v386: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:29 smithi162 ceph-mon[130941]: pgmap v387: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:29 smithi183 ceph-mon[131414]: pgmap v387: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:29 smithi145 ceph-mon[97681]: pgmap v387: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:57:31.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:30 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:57:31.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:30 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:57:31.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:30 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:57:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: pgmap v388: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:57:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:31 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: pgmap v388: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:57:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.415 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:31 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: pgmap v388: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:57:32.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:32.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:31 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:33.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:32 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:57:33.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:32 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:57:33.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:32 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:33.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:32 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:57:33.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:32 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:57:33.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:32 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:33.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:32 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:57:33.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:32 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:57:33.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:32 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:57:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:33 smithi145 ceph-mon[97681]: pgmap v389: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:34.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:33 smithi162 ceph-mon[130941]: pgmap v389: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:34.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:33 smithi183 ceph-mon[131414]: pgmap v389: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:36 smithi145 ceph-mon[97681]: pgmap v390: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:36.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:36 smithi162 ceph-mon[130941]: pgmap v390: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:36.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:36 smithi183 ceph-mon[131414]: pgmap v390: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:38.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:38 smithi145 ceph-mon[97681]: pgmap v391: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:38.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:38 smithi162 ceph-mon[130941]: pgmap v391: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:38 smithi183 ceph-mon[131414]: pgmap v391: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:40.071 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:57:40.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/1077296453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 msgr2=0x7fc1a40730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/1077296453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a40730a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009990 tx=0x7fc1a0009ca0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/1077296453 shutdown_connections 2023-08-25T04:57:40.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/1077296453 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a4113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/1077296453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a40730a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/1077296453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 0x7fc1a4111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/1077296453 >> 172.21.15.145:0/1077296453 conn(0x7fc1a40fc380 msgr2=0x7fc1a40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/1077296453 shutdown_connections 2023-08-25T04:57:40.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.307+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/1077296453 wait complete. 2023-08-25T04:57:40.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.308+0000 7fc1ab3ff700 1 Processor -- start 2023-08-25T04:57:40.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 -- start start 2023-08-25T04:57:40.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 0x7fc1a419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41a2030 con 0x7fc1a41118a0 2023-08-25T04:57:40.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41a21a0 con 0x7fc1a40735e0 2023-08-25T04:57:40.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1ab3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41a2360 con 0x7fc1a4072cc0 2023-08-25T04:57:40.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1a919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1a919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56966/0 (socket says 172.21.15.145:56966) 2023-08-25T04:57:40.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1a919b700 1 -- 172.21.15.145:0/288947445 learned_addr learned my addr 172.21.15.145:0/288947445 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:40.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1a999c700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.309+0000 7fc1a899a700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 0x7fc1a419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 -- 172.21.15.145:0/288947445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 msgr2=0x7fc1a419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 0x7fc1a419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 -- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 msgr2=0x7fc1a41a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a41a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 -- 172.21.15.145:0/288947445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00095f0 con 0x7fc1a4072cc0 2023-08-25T04:57:40.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a999c700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a41a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:57:40.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1a919b700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005910 tx=0x7fc1a00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc19a7fc700 1 -- 172.21.15.145:0/288947445 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a001d070 con 0x7fc1a4072cc0 2023-08-25T04:57:40.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc19a7fc700 1 -- 172.21.15.145:0/288947445 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a0022470 con 0x7fc1a4072cc0 2023-08-25T04:57:40.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a41a25e0 con 0x7fc1a4072cc0 2023-08-25T04:57:40.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc19a7fc700 1 -- 172.21.15.145:0/288947445 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0017650 con 0x7fc1a4072cc0 2023-08-25T04:57:40.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.310+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a9e80 con 0x7fc1a4072cc0 2023-08-25T04:57:40.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.314+0000 7fc19a7fc700 1 -- 172.21.15.145:0/288947445 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fc1a00225e0 con 0x7fc1a4072cc0 2023-08-25T04:57:40.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 Processor -- start 2023-08-25T04:57:40.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 -- start start 2023-08-25T04:57:40.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0ca0 0x5617be1ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1327b0 con 0x5617be1d0ca0 2023-08-25T04:57:40.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be123780 con 0x5617be1cac40 2023-08-25T04:57:40.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1210a0 con 0x5617be1d0380 2023-08-25T04:57:40.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.315+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56982/0 (socket says 172.21.15.145:56982) 2023-08-25T04:57:40.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 -- 172.21.15.145:0/1691598679 learned_addr learned my addr 172.21.15.145:0/1691598679 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:40.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a919b700 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0ca0 0x5617be1ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a899a700 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 -- 172.21.15.145:0/1691598679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 msgr2=0x5617be1cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 -- 172.21.15.145:0/1691598679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0ca0 msgr2=0x5617be1ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0ca0 0x5617be1ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 -- 172.21.15.145:0/1691598679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617bdf9c630 con 0x5617be1d0380 2023-08-25T04:57:40.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc19a7fc700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 0x7fc1900a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc1a999c700 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0093f0 tx=0x7fc19c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e454d11967a9909 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.316+0000 7fc19a7fc700 1 -- 172.21.15.145:0/288947445 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc1a00cccf0 con 0x7fc1a4072cc0 2023-08-25T04:57:40.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1a919b700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 0x7fc1900a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc18f7fe700 1 -- 172.21.15.145:0/1691598679 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00e030 con 0x5617be1d0380 2023-08-25T04:57:40.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc18f7fe700 1 -- 172.21.15.145:0/1691598679 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c00f040 con 0x5617be1d0380 2023-08-25T04:57:40.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc18f7fe700 1 -- 172.21.15.145:0/1691598679 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c015ab0 con 0x5617be1d0380 2023-08-25T04:57:40.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1a919b700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 0x7fc1900a2580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ca1b0 tx=0x7fc1a0022cd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 -- 172.21.15.145:0/1691598679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 msgr2=0x5617be1d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0093f0 tx=0x7fc19c009920 comp rx=0 tx=0).stop 2023-08-25T04:57:40.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 -- 172.21.15.145:0/1691598679 shutdown_connections 2023-08-25T04:57:40.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d0760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/1691598679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0ca0 0x5617be1ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.317+0000 7fc1bec72b80 1 -- 172.21.15.145:0/1691598679 >> 172.21.15.145:0/1691598679 conn(0x5617be13f0e0 msgr2=0x5617be1c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.318+0000 7fc1bec72b80 1 -- 172.21.15.145:0/1691598679 shutdown_connections 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.318+0000 7fc1bec72b80 1 -- 172.21.15.145:0/1691598679 wait complete. 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.318+0000 7fc1bec72b80 1 -- start start 2023-08-25T04:57:40.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.318+0000 7fc1bec72b80 1 Processor -- start 2023-08-25T04:57:40.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1d3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be11c990 con 0x5617be1d0380 2023-08-25T04:57:40.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be0babb0 con 0x5617be1d0ca0 2023-08-25T04:57:40.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be0ba8b0 con 0x5617be1cac40 2023-08-25T04:57:40.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1bec72b80 10 client.0 _ll_register_callbacks cb 0x5617be004a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:57:40.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:56988/0 (socket says 172.21.15.145:56988) 2023-08-25T04:57:40.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1a999c700 1 -- 172.21.15.145:0/3443858589 learned_addr learned my addr 172.21.15.145:0/3443858589 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:40.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1a899a700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1d3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.319+0000 7fc1a999c700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 -- 172.21.15.145:0/3443858589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 msgr2=0x5617be1d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1d4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 -- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 msgr2=0x5617be1d3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1d3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 -- 172.21.15.145:0/3443858589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0090d0 con 0x5617be1cac40 2023-08-25T04:57:40.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1a999c700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d50 tx=0x7fc19c006080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad65ab632625be8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc18cff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:57:40.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c026070 con 0x5617be1cac40 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617be19a1c0 con 0x5617be1cac40 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc19c004ce0 con 0x5617be1cac40 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c00e030 con 0x5617be1cac40 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1bec72b80 10 client.26206 fetch_fsmap learned FSMap version 35 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.320+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5617bdf48de0 con 0x5617be1cac40 2023-08-25T04:57:40.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617be19a1c0 con 0x5617be1cac40 2023-08-25T04:57:40.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc19c029020 con 0x5617be1cac40 2023-08-25T04:57:40.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 10 client.26206 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:57:40.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:57:40.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 20 client.26206 populate_metadata read hostname 'smithi145' 2023-08-25T04:57:40.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be1e4560 0x5617be1e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 4 client.26206 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:57:40.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617be0c2ba0 con 0x5617be1e4560 2023-08-25T04:57:40.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.321+0000 7fc1a899a700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be1e4560 0x5617be1e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.322+0000 7fc1a899a700 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be1e4560 0x5617be1e6980 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61709a88c9a23126 server_cookie=d06f9a6b5c4b0bde in_seq=0 out_seq=0 2023-08-25T04:57:40.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.322+0000 7fc18cff9700 10 client.26206 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:40.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.322+0000 7fc18cff9700 1 -- 172.21.15.145:0/3443858589 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617be0c2ba0 con 0x5617be1e4560 2023-08-25T04:57:40.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.322+0000 7fc18cff9700 10 client.26206 handle_command_reply: tid=0 2023-08-25T04:57:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.322+0000 7fc1bec72b80 1 client.26206 shutdown 2023-08-25T04:57:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 msgr2=0x5617be1d6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d50 tx=0x7fc19c006080 comp rx=0 tx=0).stop 2023-08-25T04:57:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 shutdown_connections 2023-08-25T04:57:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be1e4560 0x5617be1e6980 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1cac40 0x5617be1d6c20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/3443858589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1d3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc18cff9700 0 client.26206 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 >> 172.21.15.145:0/3443858589 conn(0x5617be13f0e0 msgr2=0x5617be1bfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 shutdown_connections 2023-08-25T04:57:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 1 -- 172.21.15.145:0/3443858589 wait complete. 2023-08-25T04:57:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.323+0000 7fc1bec72b80 20 client.26206 trim_cache size 0 max 16384 2023-08-25T04:57:40.341 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:40 smithi145 ceph-mon[97681]: pgmap v392: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:40.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.345+0000 7fc1bec72b80 1 Processor -- start 2023-08-25T04:57:40.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.345+0000 7fc1bec72b80 1 -- start start 2023-08-25T04:57:40.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 0x5617be1db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bdf9e920 con 0x5617be1cac40 2023-08-25T04:57:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1c5f00 con 0x5617be1d0ca0 2023-08-25T04:57:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1d72a0 con 0x5617be1d0380 2023-08-25T04:57:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57004/0 (socket says 172.21.15.145:57004) 2023-08-25T04:57:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 -- 172.21.15.145:0/2754953793 learned_addr learned my addr 172.21.15.145:0/2754953793 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a999c700 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 0x5617be1db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 -- 172.21.15.145:0/2754953793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 msgr2=0x5617be1cce10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T04:57:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1cce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 -- 172.21.15.145:0/2754953793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 msgr2=0x5617be1db7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 0x5617be1db7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 -- 172.21.15.145:0/2754953793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617be1eb5f0 con 0x5617be1d0380 2023-08-25T04:57:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a999c700 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 0x5617be1db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:57:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.346+0000 7fc1a899a700 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5617be1d5450 tx=0x7fc19401cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2af432bf64327b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.347+0000 7fc18ffff700 1 -- 172.21.15.145:0/2754953793 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194017040 con 0x5617be1d0380 2023-08-25T04:57:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.347+0000 7fc18ffff700 1 -- 172.21.15.145:0/2754953793 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1940119c0 con 0x5617be1d0380 2023-08-25T04:57:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.347+0000 7fc18ffff700 1 -- 172.21.15.145:0/2754953793 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400f990 con 0x5617be1d0380 2023-08-25T04:57:40.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.347+0000 7fc1bec72b80 1 -- 172.21.15.145:0/2754953793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 msgr2=0x5617be1d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.347+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5617be1d5450 tx=0x7fc19401cbe0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 -- 172.21.15.145:0/2754953793 shutdown_connections 2023-08-25T04:57:40.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1d0ca0 0x5617be1cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1cac40 0x5617be1db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/2754953793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0380 0x5617be1d67c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 -- 172.21.15.145:0/2754953793 >> 172.21.15.145:0/2754953793 conn(0x5617be13f0e0 msgr2=0x5617be1e09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 -- 172.21.15.145:0/2754953793 shutdown_connections 2023-08-25T04:57:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 -- 172.21.15.145:0/2754953793 wait complete. 2023-08-25T04:57:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 -- start start 2023-08-25T04:57:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.348+0000 7fc1bec72b80 1 Processor -- start 2023-08-25T04:57:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1d9d80 con 0x5617be1d0380 2023-08-25T04:57:40.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1d9ef0 con 0x5617be1cac40 2023-08-25T04:57:40.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617be1da0b0 con 0x5617be1d0ca0 2023-08-25T04:57:40.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1bec72b80 10 client.0 _ll_register_callbacks cb 0x7fc19c0092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1a999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:57010/0 (socket says 172.21.15.145:57010) 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1a999c700 1 -- 172.21.15.145:0/124798961 learned_addr learned my addr 172.21.15.145:0/124798961 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1a999c700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.349+0000 7fc1a899a700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.358 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 -- 172.21.15.145:0/124798961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 msgr2=0x5617be1db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1db130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 -- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 msgr2=0x5617be1db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:57:40.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.359 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 -- 172.21.15.145:0/124798961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc194009b30 con 0x5617be1d0ca0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1a999c700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc19c01ee80 tx=0x7fc19c009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e7bc1eb242b7c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc18cff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00e070 con 0x5617be1d0ca0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617be1bd8b0 con 0x5617be1d0ca0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc19c01e720 con 0x5617be1d0ca0 2023-08-25T04:57:40.360 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c026040 con 0x5617be1d0ca0 2023-08-25T04:57:40.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1bec72b80 10 client.26212 fetch_fsmap learned FSMap version 35 2023-08-25T04:57:40.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.350+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5617bdf31c60 con 0x5617be1d0ca0 2023-08-25T04:57:40.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617be1bd8b0 con 0x5617be1d0ca0 2023-08-25T04:57:40.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc19c029070 con 0x5617be1d0ca0 2023-08-25T04:57:40.361 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:57:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:57:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 20 client.26212 populate_metadata read hostname 'smithi145' 2023-08-25T04:57:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be2392c0 0x5617be23b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:57:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 4 client.26212 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:57:40.362 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617be115970 con 0x5617be2392c0 2023-08-25T04:57:40.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.351+0000 7fc1a899a700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be2392c0 0x5617be23b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:57:40.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc18cff9700 10 client.26212 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:40.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1a899a700 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be2392c0 0x5617be23b6e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f9bef3c8e96a3e server_cookie=e411038f21c0f069 in_seq=0 out_seq=0 2023-08-25T04:57:40.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc18cff9700 1 -- 172.21.15.145:0/124798961 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5617be115970 con 0x5617be2392c0 2023-08-25T04:57:40.363 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc18cff9700 10 client.26212 handle_command_reply: tid=0 2023-08-25T04:57:40.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 client.26212 shutdown 2023-08-25T04:57:40.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 msgr2=0x5617be1d9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc19c01ee80 tx=0x7fc19c009ce0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 shutdown_connections 2023-08-25T04:57:40.364 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5617be2392c0 0x5617be23b6e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617be1d0ca0 0x5617be1d9780 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617be1cac40 0x5617be1db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 --2- 172.21.15.145:0/124798961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617be1d0380 0x5617be1db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc18cff9700 0 client.26212 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.352+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 >> 172.21.15.145:0/124798961 conn(0x5617be13f0e0 msgr2=0x5617be1bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.365 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 shutdown_connections 2023-08-25T04:57:40.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1bec72b80 1 -- 172.21.15.145:0/124798961 wait complete. 2023-08-25T04:57:40.366 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1bec72b80 20 client.26212 trim_cache size 0 max 16384 2023-08-25T04:57:40.366 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:57:40.367 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:57:40.367 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:57:40.367 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:57:40.367 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:57:40.367 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:57:40.368 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:57:40.368 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:57:40.368 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:57:40.368 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:57:40.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 msgr2=0x7fc1900a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 0x7fc1900a2580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00ca1b0 tx=0x7fc1a0022cd0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.369 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 msgr2=0x7fc1a419d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:57:40.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005910 tx=0x7fc1a00047c0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 shutdown_connections 2023-08-25T04:57:40.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1a41118a0 0x7fc1a41a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1a4072cc0 0x7fc1a419d050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.370 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc1900a00c0 0x7fc1900a2580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 --2- 172.21.15.145:0/288947445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1a40735e0 0x7fc1a419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:57:40.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.353+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 >> 172.21.15.145:0/288947445 conn(0x7fc1a40fc380 msgr2=0x7fc1a40fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:57:40.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.354+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 shutdown_connections 2023-08-25T04:57:40.371 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:57:40.354+0000 7fc1ab3ff700 1 -- 172.21.15.145:0/288947445 wait complete. 2023-08-25T04:57:40.375 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:57:40.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:40 smithi162 ceph-mon[130941]: pgmap v392: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:40.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:40 smithi183 ceph-mon[131414]: pgmap v392: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:42 smithi145 ceph-mon[97681]: pgmap v393: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:42 smithi162 ceph-mon[130941]: pgmap v393: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:42.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:42 smithi183 ceph-mon[131414]: pgmap v393: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:44 smithi145 ceph-mon[97681]: pgmap v394: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:45.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:44 smithi183 ceph-mon[131414]: pgmap v394: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:45.035 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:44 smithi162 ceph-mon[130941]: pgmap v394: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:46.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:46 smithi145 ceph-mon[97681]: pgmap v395: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:46.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:46 smithi162 ceph-mon[130941]: pgmap v395: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:46.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:46 smithi183 ceph-mon[131414]: pgmap v395: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:48.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:48 smithi145 ceph-mon[97681]: pgmap v396: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:48.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:48 smithi162 ceph-mon[130941]: pgmap v396: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:48 smithi183 ceph-mon[131414]: pgmap v396: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:50.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:50 smithi145 ceph-mon[97681]: pgmap v397: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:50.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:50 smithi162 ceph-mon[130941]: pgmap v397: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:50.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:50 smithi183 ceph-mon[131414]: pgmap v397: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:52.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:52 smithi145 ceph-mon[97681]: pgmap v398: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:52.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:52 smithi162 ceph-mon[130941]: pgmap v398: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:52.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:52 smithi183 ceph-mon[131414]: pgmap v398: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:54.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:54 smithi145 ceph-mon[97681]: pgmap v399: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:54 smithi162 ceph-mon[130941]: pgmap v399: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:54.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:54 smithi183 ceph-mon[131414]: pgmap v399: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:56.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:56 smithi145 ceph-mon[97681]: pgmap v400: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:56.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:56 smithi162 ceph-mon[130941]: pgmap v400: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:56.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:56 smithi183 ceph-mon[131414]: pgmap v400: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:57:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:57:58 smithi145 ceph-mon[97681]: pgmap v401: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:57:58 smithi162 ceph-mon[130941]: pgmap v401: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:57:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:57:58 smithi183 ceph-mon[131414]: pgmap v401: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:00.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:00 smithi145 ceph-mon[97681]: pgmap v402: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:00 smithi162 ceph-mon[130941]: pgmap v402: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:00 smithi183 ceph-mon[131414]: pgmap v402: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:02 smithi145 ceph-mon[97681]: pgmap v403: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:02.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:02 smithi162 ceph-mon[130941]: pgmap v403: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:02.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:02 smithi183 ceph-mon[131414]: pgmap v403: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:04 smithi145 ceph-mon[97681]: pgmap v404: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:04.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:04 smithi162 ceph-mon[130941]: pgmap v404: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:04.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:04 smithi183 ceph-mon[131414]: pgmap v404: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:06 smithi145 ceph-mon[97681]: pgmap v405: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:06.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:06 smithi162 ceph-mon[130941]: pgmap v405: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:06 smithi183 ceph-mon[131414]: pgmap v405: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:08 smithi145 ceph-mon[97681]: pgmap v406: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:08.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:08 smithi162 ceph-mon[130941]: pgmap v406: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:08.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:08 smithi183 ceph-mon[131414]: pgmap v406: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:10 smithi145 ceph-mon[97681]: pgmap v407: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:58:10.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:58:10.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:10 smithi162 ceph-mon[130941]: pgmap v407: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:58:10.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:10 smithi183 ceph-mon[131414]: pgmap v407: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:58:10.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 -- 172.21.15.145:0/3146983573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5014105680 msgr2=0x7f5014105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 --2- 172.21.15.145:0/3146983573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5014105680 0x7f5014105a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f50040098f0 tx=0x7f5004009c00 comp rx=0 tx=0).stop 2023-08-25T04:58:10.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 -- 172.21.15.145:0/3146983573 shutdown_connections 2023-08-25T04:58:10.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 --2- 172.21.15.145:0/3146983573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 --2- 172.21.15.145:0/3146983573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5014105680 0x7f5014105a60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 --2- 172.21.15.145:0/3146983573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50140ff860 0x7f50140ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 -- 172.21.15.145:0/3146983573 >> 172.21.15.145:0/3146983573 conn(0x7f50140750e0 msgr2=0x7f50140754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.604+0000 7f5013fff700 1 -- 172.21.15.145:0/3146983573 shutdown_connections 2023-08-25T04:58:10.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.605+0000 7f5013fff700 1 -- 172.21.15.145:0/3146983573 wait complete. 2023-08-25T04:58:10.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.606+0000 7f5013fff700 1 Processor -- start 2023-08-25T04:58:10.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.606+0000 7f5013fff700 1 -- start start 2023-08-25T04:58:10.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.606+0000 7f5013fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.606+0000 7f5013fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5013fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014105680 0x7f5014198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5013fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5014198dd0 con 0x7f50140ff860 2023-08-25T04:58:10.613 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5013fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5014198f40 con 0x7f5014105680 2023-08-25T04:58:10.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5013fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5014199100 con 0x7f5014100220 2023-08-25T04:58:10.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.614 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f50127fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43452/0 (socket says 172.21.15.145:43452) 2023-08-25T04:58:10.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 -- 172.21.15.145:0/1233298350 learned_addr learned my addr 172.21.15.145:0/1233298350 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:10.615 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f50137fe700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014105680 0x7f5014198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 -- 172.21.15.145:0/1233298350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 msgr2=0x7f501419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 -- 172.21.15.145:0/1233298350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014105680 msgr2=0x7f5014198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014105680 0x7f5014198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f5012ffd700 1 -- 172.21.15.145:0/1233298350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004009520 con 0x7f50140ff860 2023-08-25T04:58:10.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.607+0000 7f50127fc700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:58:10.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f5012ffd700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f500400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f4ffbfff700 1 -- 172.21.15.145:0/1233298350 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500401c070 con 0x7f50140ff860 2023-08-25T04:58:10.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f4ffbfff700 1 -- 172.21.15.145:0/1233298350 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50040058e0 con 0x7f50140ff860 2023-08-25T04:58:10.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f4ffbfff700 1 -- 172.21.15.145:0/1233298350 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50040038e0 con 0x7f50140ff860 2023-08-25T04:58:10.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50141993e0 con 0x7f50140ff860 2023-08-25T04:58:10.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.608+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50141a56e0 con 0x7f50140ff860 2023-08-25T04:58:10.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.614+0000 7f4ffbfff700 1 -- 172.21.15.145:0/1233298350 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f5004020030 con 0x7f50140ff860 2023-08-25T04:58:10.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.614+0000 7f502cf1ab80 1 Processor -- start 2023-08-25T04:58:10.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 -- start start 2023-08-25T04:58:10.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 0x55775d05e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d05ec40 0x55775d05f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cfc67b0 con 0x55775d064500 2023-08-25T04:58:10.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cfb7780 con 0x55775d064e20 2023-08-25T04:58:10.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cfb50a0 con 0x55775d05ec40 2023-08-25T04:58:10.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f50137fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f50137fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43466/0 (socket says 172.21.15.145:43466) 2023-08-25T04:58:10.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f50137fe700 1 -- 172.21.15.145:0/922495955 learned_addr learned my addr 172.21.15.145:0/922495955 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:10.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.615+0000 7f5012ffd700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 0x55775d05e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f50127fc700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d05ec40 0x55775d05f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f4ffbfff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 0x7f4ffc0a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f4ffbfff700 1 -- 172.21.15.145:0/1233298350 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f50040cc6b0 con 0x7f50140ff860 2023-08-25T04:58:10.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 -- 172.21.15.145:0/922495955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d05ec40 msgr2=0x55775d05f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d05ec40 0x55775d05f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 -- 172.21.15.145:0/922495955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 msgr2=0x55775d0648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 -- 172.21.15.145:0/922495955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775ce30630 con 0x55775d064e20 2023-08-25T04:58:10.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f50137fe700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:58:10.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f50137fe700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 0x7f4ffc0a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.616+0000 7f5012ffd700 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 0x55775d05e700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f50040c9ac0 tx=0x7f5004009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c91c2fe38ebac8f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.617+0000 7f4ff97fa700 1 -- 172.21.15.145:0/922495955 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004099650 con 0x55775d064e20 2023-08-25T04:58:10.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.617+0000 7f4ff97fa700 1 -- 172.21.15.145:0/922495955 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5004099470 con 0x55775d064e20 2023-08-25T04:58:10.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.617+0000 7f4ff97fa700 1 -- 172.21.15.145:0/922495955 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50040a1890 con 0x55775d064e20 2023-08-25T04:58:10.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- 172.21.15.145:0/922495955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 msgr2=0x55775d05e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f50137fe700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 0x7f4ffc0a26a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f500c007680 tx=0x7f500c009040 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 0x55775d05e700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f50040c9ac0 tx=0x7f5004009260 comp rx=0 tx=0).stop 2023-08-25T04:58:10.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- 172.21.15.145:0/922495955 shutdown_connections 2023-08-25T04:58:10.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d05ec40 0x55775d05f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d0648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/922495955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d064e20 0x55775d05e700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- 172.21.15.145:0/922495955 >> 172.21.15.145:0/922495955 conn(0x55775cfd30e0 msgr2=0x55775d059940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- 172.21.15.145:0/922495955 shutdown_connections 2023-08-25T04:58:10.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- 172.21.15.145:0/922495955 wait complete. 2023-08-25T04:58:10.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.618+0000 7f502cf1ab80 1 -- start start 2023-08-25T04:58:10.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.619+0000 7f502cf1ab80 1 Processor -- start 2023-08-25T04:58:10.633 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d068030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d0686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cfb0990 con 0x55775d064e20 2023-08-25T04:58:10.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cf4ebb0 con 0x55775d05ec40 2023-08-25T04:58:10.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775cf4e8b0 con 0x55775d064500 2023-08-25T04:58:10.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f502cf1ab80 10 client.0 _ll_register_callbacks cb 0x55775ce98a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:58:10.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54172/0 (socket says 172.21.15.145:54172) 2023-08-25T04:58:10.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 -- 172.21.15.145:0/3844380627 learned_addr learned my addr 172.21.15.145:0/3844380627 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:10.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f50127fc700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d068030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f50137fe700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d0686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 -- 172.21.15.145:0/3844380627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 msgr2=0x55775d068030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d068030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 -- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 msgr2=0x55775d0686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d0686b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f5012ffd700 1 -- 172.21.15.145:0/3844380627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c011300 con 0x55775d05ec40 2023-08-25T04:58:10.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.620+0000 7f50137fe700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d0686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:58:10.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f5012ffd700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50040c9b70 tx=0x7f5004005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b2545a3cb80f23 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:58:10.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004095e50 con 0x55775d05ec40 2023-08-25T04:58:10.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f50040af060 con 0x55775d05ec40 2023-08-25T04:58:10.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55775d02e1c0 con 0x55775d05ec40 2023-08-25T04:58:10.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f500409a210 con 0x55775d05ec40 2023-08-25T04:58:10.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f502cf1ab80 10 client.26180 fetch_fsmap learned FSMap version 35 2023-08-25T04:58:10.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f50141a56e0 con 0x55775d05ec40 2023-08-25T04:58:10.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55775d02e1c0 con 0x55775d05ec40 2023-08-25T04:58:10.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.621+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f50040cc030 con 0x55775d05ec40 2023-08-25T04:58:10.646 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 10 client.26180 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:58:10.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:58:10.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 20 client.26180 populate_metadata read hostname 'smithi145' 2023-08-25T04:58:10.647 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d078160 0x55775d07a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 4 client.26180 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:58:10.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55775cf56ba0 con 0x55775d078160 2023-08-25T04:58:10.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f50127fc700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d078160 0x55775d07a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f50127fc700 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d078160 0x55775d07a580 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e66f250f7c3ec70 server_cookie=ec4f55d667df5809 in_seq=0 out_seq=0 2023-08-25T04:58:10.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.622+0000 7f4ff6ff5700 10 client.26180 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:10.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.623+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/3844380627 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55775cf56ba0 con 0x55775d078160 2023-08-25T04:58:10.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.623+0000 7f4ff6ff5700 10 client.26180 handle_command_reply: tid=0 2023-08-25T04:58:10.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.623+0000 7f502cf1ab80 1 client.26180 shutdown 2023-08-25T04:58:10.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 msgr2=0x55775d06adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50040c9b70 tx=0x7f5004005270 comp rx=0 tx=0).stop 2023-08-25T04:58:10.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 shutdown_connections 2023-08-25T04:58:10.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d0686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06adb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d078160 0x55775d07a580 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/3844380627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d068030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f4ff6ff5700 0 client.26180 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:10.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 >> 172.21.15.145:0/3844380627 conn(0x55775cfd30e0 msgr2=0x55775d059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 shutdown_connections 2023-08-25T04:58:10.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 1 -- 172.21.15.145:0/3844380627 wait complete. 2023-08-25T04:58:10.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.624+0000 7f502cf1ab80 20 client.26180 trim_cache size 0 max 16384 2023-08-25T04:58:10.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 Processor -- start 2023-08-25T04:58:10.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 -- start start 2023-08-25T04:58:10.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775ce32920 con 0x55775d064e20 2023-08-25T04:58:10.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775d055a00 con 0x55775d05ec40 2023-08-25T04:58:10.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775d05a7d0 con 0x55775d064500 2023-08-25T04:58:10.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50127fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50127fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50800/0 (socket says 172.21.15.145:50800) 2023-08-25T04:58:10.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50127fc700 1 -- 172.21.15.145:0/557994157 learned_addr learned my addr 172.21.15.145:0/557994157 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:10.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f5012ffd700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50137fe700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50127fc700 1 -- 172.21.15.145:0/557994157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 msgr2=0x55775d06a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.649+0000 7f50127fc700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f50127fc700 1 -- 172.21.15.145:0/557994157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 msgr2=0x55775d062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f50127fc700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f50127fc700 1 -- 172.21.15.145:0/557994157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775d07f190 con 0x55775d064500 2023-08-25T04:58:10.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f5012ffd700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:58:10.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f50137fe700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:58:10.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f50127fc700 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5014199bd0 tx=0x7f5008009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f973479198f5f691 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f4ff9ffb700 1 -- 172.21.15.145:0/557994157 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008017030 con 0x55775d064500 2023-08-25T04:58:10.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f4ff9ffb700 1 -- 172.21.15.145:0/557994157 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5008018040 con 0x55775d064500 2023-08-25T04:58:10.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.650+0000 7f4ff9ffb700 1 -- 172.21.15.145:0/557994157 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008004830 con 0x55775d064500 2023-08-25T04:58:10.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- 172.21.15.145:0/557994157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 msgr2=0x55775d0621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5014199bd0 tx=0x7f5008009260 comp rx=0 tx=0).stop 2023-08-25T04:58:10.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- 172.21.15.145:0/557994157 shutdown_connections 2023-08-25T04:58:10.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064e20 0x55775d062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/557994157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064500 0x55775d0621d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- 172.21.15.145:0/557994157 >> 172.21.15.145:0/557994157 conn(0x55775cfd30e0 msgr2=0x55775d074840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- 172.21.15.145:0/557994157 shutdown_connections 2023-08-25T04:58:10.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- 172.21.15.145:0/557994157 wait complete. 2023-08-25T04:58:10.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.651+0000 7f502cf1ab80 1 -- start start 2023-08-25T04:58:10.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.652+0000 7f502cf1ab80 1 Processor -- start 2023-08-25T04:58:10.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.652+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.671 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.652+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.652+0000 7f502cf1ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.672 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.652+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775d06c4d0 con 0x55775d064500 2023-08-25T04:58:10.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775d0cb4d0 con 0x55775d05ec40 2023-08-25T04:58:10.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f502cf1ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775d0cb690 con 0x55775d064e20 2023-08-25T04:58:10.673 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f502cf1ab80 10 client.0 _ll_register_callbacks cb 0x7f500c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:58:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f50137fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.674 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f50127fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:54188/0 (socket says 172.21.15.145:54188) 2023-08-25T04:58:10.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f50137fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:50810/0 (socket says 172.21.15.145:50810) 2023-08-25T04:58:10.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 -- 172.21.15.145:0/583514933 learned_addr learned my addr 172.21.15.145:0/583514933 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:10.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 -- 172.21.15.145:0/583514933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 msgr2=0x55775d0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 -- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 msgr2=0x55775d06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:10.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 -- 172.21.15.145:0/583514933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5008012170 con 0x55775d05ec40 2023-08-25T04:58:10.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f50137fe700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:58:10.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f5012ffd700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5004096580 tx=0x7f50040cc5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce123948e49e759 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:10.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.653+0000 7f4ff6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:58:10.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004099b10 con 0x55775d05ec40 2023-08-25T04:58:10.679 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f50040af070 con 0x55775d05ec40 2023-08-25T04:58:10.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55775d0518b0 con 0x55775d05ec40 2023-08-25T04:58:10.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5004095e50 con 0x55775d05ec40 2023-08-25T04:58:10.680 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 10 client.26186 fetch_fsmap learned FSMap version 35 2023-08-25T04:58:10.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55775cddcde0 con 0x55775d05ec40 2023-08-25T04:58:10.681 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55775d0518b0 con 0x55775d05ec40 2023-08-25T04:58:10.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5004015020 con 0x55775d05ec40 2023-08-25T04:58:10.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 10 client.26186 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:58:10.682 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:58:10.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 20 client.26186 populate_metadata read hostname 'smithi145' 2023-08-25T04:58:10.683 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d0cc860 0x55775d0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:10.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 4 client.26186 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:58:10.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.654+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55775cfa9970 con 0x55775d0cc860 2023-08-25T04:58:10.684 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.655+0000 7f50127fc700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d0cc860 0x55775d0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:10.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.655+0000 7f4ff6ff5700 10 client.26186 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:10.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.655+0000 7f50127fc700 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d0cc860 0x55775d0cec50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52f175f78966dbd server_cookie=6fc230672ce86da7 in_seq=0 out_seq=0 2023-08-25T04:58:10.685 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f4ff6ff5700 1 -- 172.21.15.145:0/583514933 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55775cfa9970 con 0x55775d0cc860 2023-08-25T04:58:10.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f4ff6ff5700 10 client.26186 handle_command_reply: tid=0 2023-08-25T04:58:10.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 client.26186 shutdown 2023-08-25T04:58:10.686 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 msgr2=0x55775d06b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5004096580 tx=0x7f50040cc5d0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 shutdown_connections 2023-08-25T04:58:10.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55775d0cc860 0x55775d0cec50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55775d064e20 0x55775d0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.687 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775d05ec40 0x55775d06b7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 --2- 172.21.15.145:0/583514933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775d064500 0x55775d06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f4ff6ff5700 0 client.26186 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:10.688 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 >> 172.21.15.145:0/583514933 conn(0x55775cfd30e0 msgr2=0x55775d0621d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 shutdown_connections 2023-08-25T04:58:10.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 1 -- 172.21.15.145:0/583514933 wait complete. 2023-08-25T04:58:10.689 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.656+0000 7f502cf1ab80 20 client.26186 trim_cache size 0 max 16384 2023-08-25T04:58:10.690 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:58:10.690 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:58:10.690 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:58:10.691 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:58:10.691 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:58:10.691 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:58:10.691 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:58:10.692 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:58:10.692 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:58:10.692 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:58:10.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 msgr2=0x7f4ffc0a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 0x7f4ffc0a26a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f500c007680 tx=0x7f500c009040 comp rx=0 tx=0).stop 2023-08-25T04:58:10.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 msgr2=0x7f501419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:10.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5004009230 tx=0x7f500400fa00 comp rx=0 tx=0).stop 2023-08-25T04:58:10.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 shutdown_connections 2023-08-25T04:58:10.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f4ffc0a01e0 0x7f4ffc0a26a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014105680 0x7f5014198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.695 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50140ff860 0x7f501419e6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 --2- 172.21.15.145:0/1233298350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5014100220 0x7f501419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:10.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 >> 172.21.15.145:0/1233298350 conn(0x7f50140750e0 msgr2=0x7f50140fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:10.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 shutdown_connections 2023-08-25T04:58:10.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:10.657+0000 7f5013fff700 1 -- 172.21.15.145:0/1233298350 wait complete. 2023-08-25T04:58:10.697 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:58:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:12 smithi145 ceph-mon[97681]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:12 smithi145 ceph-mon[97681]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T04:58:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:12 smithi162 ceph-mon[130941]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:12.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:12 smithi162 ceph-mon[130941]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T04:58:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:12 smithi183 ceph-mon[131414]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:58:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:12 smithi183 ceph-mon[131414]: osdmap e103: 12 total, 12 up, 12 in 2023-08-25T04:58:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:14 smithi145 ceph-mon[97681]: pgmap v410: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:14.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:14 smithi162 ceph-mon[130941]: pgmap v410: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:14 smithi183 ceph-mon[131414]: pgmap v410: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:16.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:16 smithi145 ceph-mon[97681]: pgmap v411: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:16.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:16 smithi162 ceph-mon[130941]: pgmap v411: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:16.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:16 smithi183 ceph-mon[131414]: pgmap v411: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:58:18.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:18 smithi145 ceph-mon[97681]: pgmap v412: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:18.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:18 smithi162 ceph-mon[130941]: pgmap v412: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:18.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:18 smithi183 ceph-mon[131414]: pgmap v412: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:20 smithi145 ceph-mon[97681]: pgmap v413: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:20 smithi145 ceph-mon[97681]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T04:58:20.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:20 smithi162 ceph-mon[130941]: pgmap v413: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:20.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:20 smithi162 ceph-mon[130941]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T04:58:20.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:20 smithi183 ceph-mon[131414]: pgmap v413: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:20.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:20 smithi183 ceph-mon[131414]: osdmap e104: 12 total, 12 up, 12 in 2023-08-25T04:58:22.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:22 smithi145 ceph-mon[97681]: pgmap v415: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:58:22.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:22 smithi162 ceph-mon[130941]: pgmap v415: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:58:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:22 smithi183 ceph-mon[131414]: pgmap v415: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2023-08-25T04:58:23.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:23 smithi162 ceph-mon[130941]: pgmap v416: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:23.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:23 smithi183 ceph-mon[131414]: pgmap v416: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:23.577 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:23 smithi145 ceph-mon[97681]: pgmap v416: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:25 smithi145 ceph-mon[97681]: pgmap v417: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:25 smithi162 ceph-mon[130941]: pgmap v417: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:25 smithi183 ceph-mon[131414]: pgmap v417: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:58:28.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:27 smithi145 ceph-mon[97681]: pgmap v418: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:27 smithi162 ceph-mon[130941]: pgmap v418: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:27 smithi183 ceph-mon[131414]: pgmap v418: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:30.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:29 smithi145 ceph-mon[97681]: pgmap v419: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:29 smithi162 ceph-mon[130941]: pgmap v419: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:29 smithi183 ceph-mon[131414]: pgmap v419: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:58:32.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:31 smithi145 ceph-mon[97681]: pgmap v420: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:58:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:31 smithi162 ceph-mon[130941]: pgmap v420: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:58:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:31 smithi183 ceph-mon[131414]: pgmap v420: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:58:33.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:32 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:58:33.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:32 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:58:33.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:32 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:58:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:33 smithi145 ceph-mon[97681]: pgmap v421: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:34.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:33 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:58:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:33 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:58:34.079 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:33 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:58:34.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:33 smithi162 ceph-mon[130941]: pgmap v421: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:33 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:58:34.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:33 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:58:34.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:33 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:58:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:33 smithi183 ceph-mon[131414]: pgmap v421: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:33 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:58:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:33 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:58:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:33 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:58:36.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:35 smithi145 ceph-mon[97681]: pgmap v422: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:35 smithi162 ceph-mon[130941]: pgmap v422: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:35 smithi183 ceph-mon[131414]: pgmap v422: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:38.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:37 smithi145 ceph-mon[97681]: pgmap v423: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:38.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:37 smithi162 ceph-mon[130941]: pgmap v423: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:38.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:37 smithi183 ceph-mon[131414]: pgmap v423: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:39 smithi162 ceph-mon[130941]: pgmap v424: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:39 smithi183 ceph-mon[131414]: pgmap v424: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:40.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:39 smithi145 ceph-mon[97681]: pgmap v424: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:40.699 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:58:40.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 -- 172.21.15.145:0/2560629296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c81068a0 msgr2=0x7f41c8106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/2560629296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c81068a0 0x7f41c8106c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009990 tx=0x7f41b8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 -- 172.21.15.145:0/2560629296 shutdown_connections 2023-08-25T04:58:40.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/2560629296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c8101250 0x7f41c810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/2560629296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c81068a0 0x7f41c8106c80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/2560629296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 -- 172.21.15.145:0/2560629296 >> 172.21.15.145:0/2560629296 conn(0x7f41c80fc380 msgr2=0x7f41c80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 -- 172.21.15.145:0/2560629296 shutdown_connections 2023-08-25T04:58:40.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.911+0000 7f41ce3ff700 1 -- 172.21.15.145:0/2560629296 wait complete. 2023-08-25T04:58:40.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 Processor -- start 2023-08-25T04:58:40.916 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 -- start start 2023-08-25T04:58:40.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c81068a0 0x7f41c81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8199170 con 0x7f41c8100890 2023-08-25T04:58:40.917 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c81992e0 con 0x7f41c81068a0 2023-08-25T04:58:40.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.912+0000 7f41ce3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c81994a0 con 0x7f41c8101250 2023-08-25T04:58:40.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c819e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43958/0 (socket says 172.21.15.145:43958) 2023-08-25T04:58:40.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49654/0 (socket says 172.21.15.145:49654) 2023-08-25T04:58:40.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c7fff700 1 -- 172.21.15.145:0/4159426410 learned_addr learned my addr 172.21.15.145:0/4159426410 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:40.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41cc99c700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c81068a0 0x7f41c81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 -- 172.21.15.145:0/4159426410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c81068a0 msgr2=0x7f41c81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c81068a0 0x7f41c81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 -- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 msgr2=0x7f41c819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.927 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 -- 172.21.15.145:0/4159426410 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80095f0 con 0x7f41c8101250 2023-08-25T04:58:40.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.913+0000 7f41c77fe700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.928 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.914+0000 7f41c57fa700 1 -- 172.21.15.145:0/4159426410 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00c9c0 con 0x7f41c8101250 2023-08-25T04:58:40.929 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.914+0000 7f41c57fa700 1 -- 172.21.15.145:0/4159426410 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc010430 con 0x7f41c8101250 2023-08-25T04:58:40.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.914+0000 7f41c57fa700 1 -- 172.21.15.145:0/4159426410 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc018680 con 0x7f41c8101250 2023-08-25T04:58:40.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.914+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c8199780 con 0x7f41c8101250 2023-08-25T04:58:40.930 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.914+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c81a5940 con 0x7f41c8101250 2023-08-25T04:58:40.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.918+0000 7f41c57fa700 1 -- 172.21.15.145:0/4159426410 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f41bc010860 con 0x7f41c8101250 2023-08-25T04:58:40.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.919+0000 7f41c57fa700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 0x7f41b00a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.919+0000 7f41c57fa700 1 -- 172.21.15.145:0/4159426410 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f41bc014070 con 0x7f41c8101250 2023-08-25T04:58:40.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.919+0000 7f41c7fff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 0x7f41b00a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 Processor -- start 2023-08-25T04:58:40.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41c7fff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 0x7f41b00a2600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006010 tx=0x7f41b800b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 -- start start 2023-08-25T04:58:40.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730014760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f73000e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72ff767b0 con 0x55f730014ca0 2023-08-25T04:58:40.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72ff67780 con 0x55f730014380 2023-08-25T04:58:40.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.920+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72ff650a0 con 0x55f73000ec40 2023-08-25T04:58:40.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49660/0 (socket says 172.21.15.145:49660) 2023-08-25T04:58:40.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41c77fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f73000e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 -- 172.21.15.145:0/2021957626 learned_addr learned my addr 172.21.15.145:0/2021957626 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:40.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41c7fff700 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730014760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 -- 172.21.15.145:0/2021957626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 msgr2=0x55f730014760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730014760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 -- 172.21.15.145:0/2021957626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 msgr2=0x55f73000e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f73000e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 -- 172.21.15.145:0/2021957626 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72fde0630 con 0x55f73000ec40 2023-08-25T04:58:40.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.921+0000 7f41cc99c700 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c0011600 tx=0x7f41c0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b9dc17f75c4e1d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41ae7fc700 1 -- 172.21.15.145:0/2021957626 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c000e040 con 0x55f73000ec40 2023-08-25T04:58:40.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41ae7fc700 1 -- 172.21.15.145:0/2021957626 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c000f040 con 0x55f73000ec40 2023-08-25T04:58:40.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41ae7fc700 1 -- 172.21.15.145:0/2021957626 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0004800 con 0x55f73000ec40 2023-08-25T04:58:40.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2021957626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 msgr2=0x55f73000f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c0011600 tx=0x7f41c0009920 comp rx=0 tx=0).stop 2023-08-25T04:58:40.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2021957626 shutdown_connections 2023-08-25T04:58:40.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f73000f0a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730014760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.922+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2021957626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f73000e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.923+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2021957626 >> 172.21.15.145:0/2021957626 conn(0x55f72ff830e0 msgr2=0x55f730009410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.923+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2021957626 shutdown_connections 2023-08-25T04:58:40.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.923+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2021957626 wait complete. 2023-08-25T04:58:40.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.923+0000 7f41e1d2cb80 1 -- start start 2023-08-25T04:58:40.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.923+0000 7f41e1d2cb80 1 Processor -- start 2023-08-25T04:58:40.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f73000ec40 0x55f73001ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730018500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72ff60990 con 0x55f730014ca0 2023-08-25T04:58:40.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72fefebb0 con 0x55f73000ec40 2023-08-25T04:58:40.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72fefe8b0 con 0x55f730014380 2023-08-25T04:58:40.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41e1d2cb80 10 client.0 _ll_register_callbacks cb 0x55f72fe48a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:58:40.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41c77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.947 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41c77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49674/0 (socket says 172.21.15.145:49674) 2023-08-25T04:58:40.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41c77fe700 1 -- 172.21.15.145:0/2034147732 learned_addr learned my addr 172.21.15.145:0/2034147732 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:40.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41cc99c700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730018500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.924+0000 7f41c7fff700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f73000ec40 0x55f73001ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 -- 172.21.15.145:0/2034147732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f73000ec40 msgr2=0x55f73001ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f73000ec40 0x55f73001ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 -- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 msgr2=0x55f730018500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730018500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 -- 172.21.15.145:0/2034147732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c00112b0 con 0x55f730014380 2023-08-25T04:58:40.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41c77fe700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41bc098b40 tx=0x7f41bc098d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f169ac36dd3f81 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:58:40.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0992a0 con 0x55f730014380 2023-08-25T04:58:40.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f41bc09cc70 con 0x55f730014380 2023-08-25T04:58:40.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72ffde1c0 con 0x55f730014380 2023-08-25T04:58:40.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41bc0a0c50 con 0x55f730014380 2023-08-25T04:58:40.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41e1d2cb80 10 client.26242 fetch_fsmap learned FSMap version 35 2023-08-25T04:58:40.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.925+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f72fd8cde0 con 0x55f730014380 2023-08-25T04:58:40.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f72ffde1c0 con 0x55f730014380 2023-08-25T04:58:40.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41bc094c30 con 0x55f730014380 2023-08-25T04:58:40.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 10 client.26242 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:58:40.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:58:40.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 20 client.26242 populate_metadata read hostname 'smithi145' 2023-08-25T04:58:40.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f730028680 0x55f73002aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 4 client.26242 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:58:40.955 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f72ff06ba0 con 0x55f730028680 2023-08-25T04:58:40.955 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.926+0000 7f41c7fff700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f730028680 0x55f73002aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.955 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.927+0000 7f41a77fe700 10 client.26242 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:40.955 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.927+0000 7f41c7fff700 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f730028680 0x55f73002aaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7681d1dcb0b0b0 server_cookie=73fe6e4539fb2fd1 in_seq=0 out_seq=0 2023-08-25T04:58:40.955 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.927+0000 7f41a77fe700 1 -- 172.21.15.145:0/2034147732 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f72ff06ba0 con 0x55f730028680 2023-08-25T04:58:40.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.927+0000 7f41a77fe700 10 client.26242 handle_command_reply: tid=0 2023-08-25T04:58:40.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 client.26242 shutdown 2023-08-25T04:58:40.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 msgr2=0x55f730017e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41bc098b40 tx=0x7f41bc098d40 comp rx=0 tx=0).stop 2023-08-25T04:58:40.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 shutdown_connections 2023-08-25T04:58:40.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730018500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f73000ec40 0x55f73001ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f730028680 0x55f73002aaa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/2034147732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014380 0x55f730017e80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41a77fe700 0 client.26242 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:40.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 >> 172.21.15.145:0/2034147732 conn(0x55f72ff830e0 msgr2=0x55f730003b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 shutdown_connections 2023-08-25T04:58:40.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/2034147732 wait complete. 2023-08-25T04:58:40.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.928+0000 7f41e1d2cb80 20 client.26242 trim_cache size 0 max 16384 2023-08-25T04:58:40.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.951+0000 7f41e1d2cb80 1 Processor -- start 2023-08-25T04:58:40.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 -- start start 2023-08-25T04:58:40.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f73001a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730011ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72fde2920 con 0x55f730014ca0 2023-08-25T04:58:40.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f730005a20 con 0x55f730014380 2023-08-25T04:58:40.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f730005ca0 con 0x55f73000ec40 2023-08-25T04:58:40.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41cc99c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730011ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49686/0 (socket says 172.21.15.145:49686) 2023-08-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 -- 172.21.15.145:0/4158599747 learned_addr learned my addr 172.21.15.145:0/4158599747 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c77fe700 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f73001a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 -- 172.21.15.145:0/4158599747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 msgr2=0x55f73001a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f73001a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 -- 172.21.15.145:0/4158599747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 msgr2=0x55f730011ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730011ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.952+0000 7f41c7fff700 1 -- 172.21.15.145:0/4158599747 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72fd75c60 con 0x55f73000ec40 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.953+0000 7f41c7fff700 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f41b800b6f0 tx=0x7f41b8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687991e8e8d01bc3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.953+0000 7f41aeffd700 1 -- 172.21.15.145:0/4158599747 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b802b040 con 0x55f73000ec40 2023-08-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.953+0000 7f41aeffd700 1 -- 172.21.15.145:0/4158599747 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41b802c040 con 0x55f73000ec40 2023-08-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.953+0000 7f41aeffd700 1 -- 172.21.15.145:0/4158599747 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80049a0 con 0x55f73000ec40 2023-08-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/4158599747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 msgr2=0x55f730017690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f41b800b6f0 tx=0x7f41b8020000 comp rx=0 tx=0).stop 2023-08-25T04:58:40.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/4158599747 shutdown_connections 2023-08-25T04:58:40.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f730014ca0 0x55f730011ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73000ec40 0x55f730017690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/4158599747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f73001a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/4158599747 >> 172.21.15.145:0/4158599747 conn(0x55f72ff830e0 msgr2=0x55f730024a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/4158599747 shutdown_connections 2023-08-25T04:58:40.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/4158599747 wait complete. 2023-08-25T04:58:40.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 -- start start 2023-08-25T04:58:40.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.954+0000 7f41e1d2cb80 1 Processor -- start 2023-08-25T04:58:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f73000ec40 0x55f73001f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730010270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7300110a0 con 0x55f73000ec40 2023-08-25T04:58:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73007c4b0 con 0x55f730014380 2023-08-25T04:58:40.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73007c5f0 con 0x55f730014ca0 2023-08-25T04:58:40.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41e1d2cb80 10 client.0 _ll_register_callbacks cb 0x7f41bc0987a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:58:40.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41cc99c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41cc99c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:49700/0 (socket says 172.21.15.145:49700) 2023-08-25T04:58:40.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41cc99c700 1 -- 172.21.15.145:0/859981558 learned_addr learned my addr 172.21.15.145:0/859981558 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:58:40.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.955+0000 7f41c7fff700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f73000ec40 0x55f73001f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41c77fe700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730010270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 -- 172.21.15.145:0/859981558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 msgr2=0x55f730010270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730010270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 -- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f73000ec40 msgr2=0x55f73001f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:58:40.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f73000ec40 0x55f73001f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 -- 172.21.15.145:0/859981558 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c810e060 con 0x55f730014ca0 2023-08-25T04:58:40.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41cc99c700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000f80 tx=0x7f41c00041c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30779c5c3796fb9b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:58:40.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:58:40.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c000e040 con 0x55f730014ca0 2023-08-25T04:58:40.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f41c00092e0 con 0x55f730014ca0 2023-08-25T04:58:40.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.956+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7300018b0 con 0x55f730014ca0 2023-08-25T04:58:40.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c0007500 con 0x55f730014ca0 2023-08-25T04:58:40.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 10 client.26248 fetch_fsmap learned FSMap version 35 2023-08-25T04:58:40.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f72fdb3250 con 0x55f730014ca0 2023-08-25T04:58:40.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7300018b0 con 0x55f730014ca0 2023-08-25T04:58:40.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f41c0027800 con 0x55f730014ca0 2023-08-25T04:58:40.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 10 client.26248 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:58:40.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:58:40.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 20 client.26248 populate_metadata read hostname 'smithi145' 2023-08-25T04:58:40.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f73007dc80 0x55f7300800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:58:40.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 4 client.26248 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:58:40.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f72ff59970 con 0x55f73007dc80 2023-08-25T04:58:40.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.957+0000 7f41c7fff700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f73007dc80 0x55f7300800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:58:40.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41a77fe700 10 client.26248 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:40.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41c7fff700 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f73007dc80 0x55f7300800a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a1bb25cb880b1f server_cookie=71e1201f9e8524a9 in_seq=0 out_seq=0 2023-08-25T04:58:40.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41a77fe700 1 -- 172.21.15.145:0/859981558 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f72ff59970 con 0x55f73007dc80 2023-08-25T04:58:40.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41a77fe700 10 client.26248 handle_command_reply: tid=0 2023-08-25T04:58:40.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 client.26248 shutdown 2023-08-25T04:58:40.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 msgr2=0x55f730010960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000f80 tx=0x7f41c00041c0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 shutdown_connections 2023-08-25T04:58:40.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55f73007dc80 0x55f7300800a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f730014ca0 0x55f730010960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f73000ec40 0x55f73001f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 --2- 172.21.15.145:0/859981558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f730014380 0x55f730010270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 >> 172.21.15.145:0/859981558 conn(0x55f72ff830e0 msgr2=0x55f730011ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.958+0000 7f41a77fe700 0 client.26248 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:58:40.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 shutdown_connections 2023-08-25T04:58:40.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41e1d2cb80 1 -- 172.21.15.145:0/859981558 wait complete. 2023-08-25T04:58:40.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41e1d2cb80 20 client.26248 trim_cache size 0 max 16384 2023-08-25T04:58:40.988 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:58:40.989 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:58:40.989 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:58:40.989 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:58:40.989 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:58:40.989 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:58:40.990 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:58:40.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:58:40.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:58:40.990 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:58:40.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 msgr2=0x7f41b00a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.991 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 0x7f41b00a2600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006010 tx=0x7f41b800b260 comp rx=0 tx=0).stop 2023-08-25T04:58:40.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 msgr2=0x7f41c819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:58:40.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.959+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00ed00 comp rx=0 tx=0).stop 2023-08-25T04:58:40.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 shutdown_connections 2023-08-25T04:58:40.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f41b00a0140 0x7f41b00a2600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41c81068a0 0x7f41c81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41c8100890 0x7f41c819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 --2- 172.21.15.145:0/4159426410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41c8101250 0x7f41c819ef00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:58:40.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 >> 172.21.15.145:0/4159426410 conn(0x7f41c80fc380 msgr2=0x7f41c810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:58:40.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 shutdown_connections 2023-08-25T04:58:40.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:58:40.960+0000 7f41ce3ff700 1 -- 172.21.15.145:0/4159426410 wait complete. 2023-08-25T04:58:40.994 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:58:42.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:41 smithi162 ceph-mon[130941]: pgmap v425: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:42.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:41 smithi183 ceph-mon[131414]: pgmap v425: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:41 smithi145 ceph-mon[97681]: pgmap v425: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:44.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:43 smithi162 ceph-mon[130941]: pgmap v426: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:44.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:43 smithi183 ceph-mon[131414]: pgmap v426: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:44.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:43 smithi145 ceph-mon[97681]: pgmap v426: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:46.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:45 smithi162 ceph-mon[130941]: pgmap v427: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:45 smithi183 ceph-mon[131414]: pgmap v427: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:46.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:45 smithi145 ceph-mon[97681]: pgmap v427: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:48.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:47 smithi162 ceph-mon[130941]: pgmap v428: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:48.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:47 smithi183 ceph-mon[131414]: pgmap v428: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:48.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:47 smithi145 ceph-mon[97681]: pgmap v428: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:50.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:49 smithi162 ceph-mon[130941]: pgmap v429: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:50.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:49 smithi183 ceph-mon[131414]: pgmap v429: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:50.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:49 smithi145 ceph-mon[97681]: pgmap v429: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:51 smithi162 ceph-mon[130941]: pgmap v430: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:51 smithi183 ceph-mon[131414]: pgmap v430: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:52.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:51 smithi145 ceph-mon[97681]: pgmap v430: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:54.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:53 smithi162 ceph-mon[130941]: pgmap v431: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:54.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:53 smithi183 ceph-mon[131414]: pgmap v431: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:54.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:53 smithi145 ceph-mon[97681]: pgmap v431: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:56.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:55 smithi162 ceph-mon[130941]: pgmap v432: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:55 smithi183 ceph-mon[131414]: pgmap v432: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:56.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:55 smithi145 ceph-mon[97681]: pgmap v432: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:58:58.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:57 smithi162 ceph-mon[130941]: pgmap v433: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:57 smithi183 ceph-mon[131414]: pgmap v433: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:58:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:57 smithi145 ceph-mon[97681]: pgmap v433: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:00.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:58:59 smithi162 ceph-mon[130941]: pgmap v434: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:00.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:58:59 smithi183 ceph-mon[131414]: pgmap v434: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:00.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:58:59 smithi145 ceph-mon[97681]: pgmap v434: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:02.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:01 smithi162 ceph-mon[130941]: pgmap v435: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:02.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:01 smithi183 ceph-mon[131414]: pgmap v435: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:01 smithi145 ceph-mon[97681]: pgmap v435: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:04.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:03 smithi162 ceph-mon[130941]: pgmap v436: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:04.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:03 smithi183 ceph-mon[131414]: pgmap v436: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:03 smithi145 ceph-mon[97681]: pgmap v436: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:06.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:05 smithi162 ceph-mon[130941]: pgmap v437: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:06.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:05 smithi183 ceph-mon[131414]: pgmap v437: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:06.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:05 smithi145 ceph-mon[97681]: pgmap v437: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:08.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:07 smithi162 ceph-mon[130941]: pgmap v438: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:08.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:07 smithi183 ceph-mon[131414]: pgmap v438: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:08.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:07 smithi145 ceph-mon[97681]: pgmap v438: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:10.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:09 smithi162 ceph-mon[130941]: pgmap v439: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:59:10.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:09 smithi183 ceph-mon[131414]: pgmap v439: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:59:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:09 smithi145 ceph-mon[97681]: pgmap v439: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T04:59:10.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:59:11.238 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 -- 172.21.15.145:0/728341001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 msgr2=0x7fa1d8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/728341001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8108e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009990 tx=0x7fa1c8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 -- 172.21.15.145:0/728341001 shutdown_connections 2023-08-25T04:59:12.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/728341001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 0x7fa1d810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/728341001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8108e80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/728341001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 0x7fa1d8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 -- 172.21.15.145:0/728341001 >> 172.21.15.145:0/728341001 conn(0x7fa1d80fe580 msgr2=0x7fa1d81009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 -- 172.21.15.145:0/728341001 shutdown_connections 2023-08-25T04:59:12.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.237+0000 7fa1ddfff700 1 -- 172.21.15.145:0/728341001 wait complete. 2023-08-25T04:59:12.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:11 smithi183 ceph-mon[131414]: pgmap v440: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:12.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:11 smithi183 ceph-mon[131414]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T04:59:12.293 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:11 smithi162 ceph-mon[130941]: pgmap v440: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:12.293 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:11 smithi162 ceph-mon[130941]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T04:59:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.238+0000 7fa1ddfff700 1 Processor -- start 2023-08-25T04:59:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.238+0000 7fa1ddfff700 1 -- start start 2023-08-25T04:59:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 0x7fa1d819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 0x7fa1d819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8199080 con 0x7fa1d8103450 2023-08-25T04:59:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81991f0 con 0x7fa1d8102a90 2023-08-25T04:59:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1ddfff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81993b0 con 0x7fa1d8108aa0 2023-08-25T04:59:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 0x7fa1d819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:37522/0 (socket says 172.21.15.145:37522) 2023-08-25T04:59:12.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1d7fff700 1 -- 172.21.15.145:0/2517720129 learned_addr learned my addr 172.21.15.145:0/2517720129 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:12.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.239+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 0x7fa1d819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 -- 172.21.15.145:0/2517720129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 msgr2=0x7fa1d819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 0x7fa1d819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 -- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 msgr2=0x7fa1d819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 0x7fa1d819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 -- 172.21.15.145:0/2517720129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80095f0 con 0x7fa1d8108aa0 2023-08-25T04:59:12.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d7fff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007ee0 tx=0x7fa1d000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d4ff9700 1 -- 172.21.15.145:0/2517720129 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0017070 con 0x7fa1d8108aa0 2023-08-25T04:59:12.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d4ff9700 1 -- 172.21.15.145:0/2517720129 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d000f040 con 0x7fa1d8108aa0 2023-08-25T04:59:12.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d8199690 con 0x7fa1d8108aa0 2023-08-25T04:59:12.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1d4ff9700 1 -- 172.21.15.145:0/2517720129 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0005580 con 0x7fa1d8108aa0 2023-08-25T04:59:12.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.240+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d81a5990 con 0x7fa1d8108aa0 2023-08-25T04:59:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.244+0000 7fa1d4ff9700 1 -- 172.21.15.145:0/2517720129 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fa1d000db20 con 0x7fa1d8108aa0 2023-08-25T04:59:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.245+0000 7fa1f18b1b80 1 Processor -- start 2023-08-25T04:59:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.245+0000 7fa1f18b1b80 1 -- start start 2023-08-25T04:59:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350287b0 con 0x55d7350c0c40 2023-08-25T04:59:12.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d735019780 con 0x55d7350c6380 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350170a0 con 0x55d7350c6ca0 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:37532/0 (socket says 172.21.15.145:37532) 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1850467013 learned_addr learned my addr 172.21.15.145:0/1850467013 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d4ff9700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 0x7fa1c00a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1850467013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 msgr2=0x55d7350c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1850467013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 msgr2=0x55d7350c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1850467013 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d734e92630 con 0x55d7350c6ca0 2023-08-25T04:59:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.246+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T04:59:12.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1d4ff9700 1 -- 172.21.15.145:0/2517720129 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fa1d00cb070 con 0x7fa1d8108aa0 2023-08-25T04:59:12.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1d7fff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 0x7fa1c00a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005e80 tx=0x7fa1c8009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f0832caaa9c06b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1bdffb700 1 -- 172.21.15.145:0/1850467013 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c801d050 con 0x55d7350c6ca0 2023-08-25T04:59:12.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1bdffb700 1 -- 172.21.15.145:0/1850467013 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c801e040 con 0x55d7350c6ca0 2023-08-25T04:59:12.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.247+0000 7fa1bdffb700 1 -- 172.21.15.145:0/1850467013 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80052f0 con 0x55d7350c6ca0 2023-08-25T04:59:12.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1850467013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 msgr2=0x55d7350c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1d7fff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 0x7fa1c00a2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0094110 tx=0x7fa1d0007f10 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005e80 tx=0x7fa1c8009ee0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1850467013 shutdown_connections 2023-08-25T04:59:12.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1850467013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350c0700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1850467013 >> 172.21.15.145:0/1850467013 conn(0x55d7350350e0 msgr2=0x55d7350bb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1850467013 shutdown_connections 2023-08-25T04:59:12.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.248+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1850467013 wait complete. 2023-08-25T04:59:12.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.249+0000 7fa1f18b1b80 1 -- start start 2023-08-25T04:59:12.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.249+0000 7fa1f18b1b80 1 Processor -- start 2023-08-25T04:59:12.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d735012990 con 0x55d7350c0c40 2023-08-25T04:59:12.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d734fb0bb0 con 0x55d7350c6380 2023-08-25T04:59:12.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d734fb08b0 con 0x55d7350c6ca0 2023-08-25T04:59:12.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1f18b1b80 10 client.0 _ll_register_callbacks cb 0x55d734efaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:59:12.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53592/0 (socket says 172.21.15.145:53592) 2023-08-25T04:59:12.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/649248253 learned_addr learned my addr 172.21.15.145:0/649248253 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:12.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.250+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 -- 172.21.15.145:0/649248253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 msgr2=0x55d7350ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 -- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 msgr2=0x55d7350ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 -- 172.21.15.145:0/649248253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d8110260 con 0x55d7350c6380 2023-08-25T04:59:12.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c9f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80098e0 tx=0x7fa1c80036e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e71efe8fab3cc330 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2023-08-25T04:59:12.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80052d0 con 0x55d7350c6380 2023-08-25T04:59:12.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7350901c0 con 0x55d7350c6380 2023-08-25T04:59:12.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.251+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mon.2 v2:172.21.15.162:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fa1c800be80 con 0x55d7350c6380 2023-08-25T04:59:12.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.252+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c8005980 con 0x55d7350c6380 2023-08-25T04:59:12.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.252+0000 7fa1f18b1b80 10 client.26216 fetch_fsmap learned FSMap version 35 2023-08-25T04:59:12.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.252+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d734e3ede0 con 0x55d7350c6380 2023-08-25T04:59:12.328 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.252+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mon.2 v2:172.21.15.162:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7350901c0 con 0x55d7350c6380 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.252+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mon.2 v2:172.21.15.162:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1c802c430 con 0x55d7350c6380 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 20 client.26216 populate_metadata read hostname 'smithi145' 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7350da1b0 0x55d7350dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.329 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 4 client.26216 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:59:12.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d734fb8ba0 con 0x55d7350da1b0 2023-08-25T04:59:12.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7350da1b0 0x55d7350dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1b6ffd700 10 client.26216 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:12.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.253+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7350da1b0 0x55d7350dc5a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2177416632f0bf1c server_cookie=17c5568eda568c5f in_seq=0 out_seq=0 2023-08-25T04:59:12.330 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/649248253 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d734fb8ba0 con 0x55d7350da1b0 2023-08-25T04:59:12.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1b6ffd700 10 client.26216 handle_command_reply: tid=0 2023-08-25T04:59:12.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 client.26216 shutdown 2023-08-25T04:59:12.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 msgr2=0x55d7350c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c9f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80098e0 tx=0x7fa1c80036e0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.331 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 shutdown_connections 2023-08-25T04:59:12.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d7350da1b0 0x55d7350dc5a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c6ca0 0x55d7350ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c0c40 0x55d7350ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/649248253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350c9f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.332 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1b6ffd700 0 client.26216 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:12.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.254+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 >> 172.21.15.145:0/649248253 conn(0x55d7350350e0 msgr2=0x55d7350b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.255+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 shutdown_connections 2023-08-25T04:59:12.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.255+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/649248253 wait complete. 2023-08-25T04:59:12.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.255+0000 7fa1f18b1b80 20 client.26216 trim_cache size 0 max 16384 2023-08-25T04:59:12.333 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.275+0000 7fa1f18b1b80 1 Processor -- start 2023-08-25T04:59:12.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.275+0000 7fa1f18b1b80 1 -- start start 2023-08-25T04:59:12.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6ca0 0x55d7350c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.334 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d734e94920 con 0x55d7350c6ca0 2023-08-25T04:59:12.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350cb390 con 0x55d7350c6380 2023-08-25T04:59:12.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350cd2a0 con 0x55d7350c0c40 2023-08-25T04:59:12.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:37552/0 (socket says 172.21.15.145:37552) 2023-08-25T04:59:12.335 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/3944013124 learned_addr learned my addr 172.21.15.145:0/3944013124 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:12.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6ca0 0x55d7350c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.276+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.336 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/3944013124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 msgr2=0x55d7350cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/3944013124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6ca0 msgr2=0x55d7350c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6ca0 0x55d7350c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.337 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/3944013124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d734e27c60 con 0x55d7350c0c40 2023-08-25T04:59:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc000c00 tx=0x7fa1cc0073d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb8bba4756d22d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1be7fc700 1 -- 172.21.15.145:0/3944013124 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc018070 con 0x55d7350c0c40 2023-08-25T04:59:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1be7fc700 1 -- 172.21.15.145:0/3944013124 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1cc028030 con 0x55d7350c0c40 2023-08-25T04:59:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.277+0000 7fa1be7fc700 1 -- 172.21.15.145:0/3944013124 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc005550 con 0x55d7350c0c40 2023-08-25T04:59:12.338 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/3944013124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 msgr2=0x55d7350c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc000c00 tx=0x7fa1cc0073d0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/3944013124 shutdown_connections 2023-08-25T04:59:12.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6ca0 0x55d7350c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c0700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.339 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/3944013124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6380 0x55d7350cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/3944013124 >> 172.21.15.145:0/3944013124 conn(0x55d7350350e0 msgr2=0x55d7350c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/3944013124 shutdown_connections 2023-08-25T04:59:12.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.278+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/3944013124 wait complete. 2023-08-25T04:59:12.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.279+0000 7fa1f18b1b80 1 -- start start 2023-08-25T04:59:12.340 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.279+0000 7fa1f18b1b80 1 Processor -- start 2023-08-25T04:59:12.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 0x55d7350c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6ca0 0x55d7350c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350c3b70 con 0x55d7350c6380 2023-08-25T04:59:12.341 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350c2880 con 0x55d7350c6ca0 2023-08-25T04:59:12.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7350c2a40 con 0x55d7350c0c40 2023-08-25T04:59:12.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1f18b1b80 10 client.0 _ll_register_callbacks cb 0x7fa1c8005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:59:12.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:37566/0 (socket says 172.21.15.145:37566) 2023-08-25T04:59:12.342 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d6ffd700 1 -- 172.21.15.145:0/1420449928 learned_addr learned my addr 172.21.15.145:0/1420449928 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:12.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 0x55d7350c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6ca0 0x55d7350c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1420449928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 msgr2=0x55d7350c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.343 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1420449928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6ca0 msgr2=0x55d7350c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:12.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6ca0 0x55d7350c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d77fe700 1 -- 172.21.15.145:0/1420449928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc01a040 con 0x55d7350c6380 2023-08-25T04:59:12.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.280+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:59:12.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1d77fe700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 0x55d7350c1e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005e80 tx=0x7fa1c8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4048e937e46c1914 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:12.344 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2023-08-25T04:59:12.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c801d070 con 0x55d7350c6380 2023-08-25T04:59:12.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7350b38b0 con 0x55d7350c6380 2023-08-25T04:59:12.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fa1c8000e40 con 0x55d7350c6380 2023-08-25T04:59:12.345 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c8009290 con 0x55d7350c6380 2023-08-25T04:59:12.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1f18b1b80 10 client.16227 fetch_fsmap learned FSMap version 35 2023-08-25T04:59:12.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d734e65250 con 0x55d7350c6380 2023-08-25T04:59:12.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.281+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7350b38b0 con 0x55d7350c6380 2023-08-25T04:59:12.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1c800f920 con 0x55d7350c6380 2023-08-25T04:59:12.346 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:59:12.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:59:12.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 20 client.16227 populate_metadata read hostname 'smithi145' 2023-08-25T04:59:12.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d73512edb0 0x55d7351311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:12.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 4 client.16227 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:59:12.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d73500b970 con 0x55d73512edb0 2023-08-25T04:59:12.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.282+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d73512edb0 0x55d7351311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:12.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1d6ffd700 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d73512edb0 0x55d7351311d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22c729d9f826d91 server_cookie=fef966c2d3b1550b in_seq=0 out_seq=0 2023-08-25T04:59:12.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1b6ffd700 10 client.16227 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:12.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1b6ffd700 1 -- 172.21.15.145:0/1420449928 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d73500b970 con 0x55d73512edb0 2023-08-25T04:59:12.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1b6ffd700 10 client.16227 handle_command_reply: tid=0 2023-08-25T04:59:12.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 client.16227 shutdown 2023-08-25T04:59:12.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 msgr2=0x55d7350c1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 0x55d7350c1e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005e80 tx=0x7fa1c8009ca0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 shutdown_connections 2023-08-25T04:59:12.349 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7350c6ca0 0x55d7350c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x55d73512edb0 0x55d7351311d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7350c0c40 0x55d7350c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 --2- 172.21.15.145:0/1420449928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7350c6380 0x55d7350c1e00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1b6ffd700 0 client.16227 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:12.350 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 >> 172.21.15.145:0/1420449928 conn(0x55d7350350e0 msgr2=0x55d7350c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 shutdown_connections 2023-08-25T04:59:12.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.283+0000 7fa1f18b1b80 1 -- 172.21.15.145:0/1420449928 wait complete. 2023-08-25T04:59:12.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1f18b1b80 20 client.16227 trim_cache size 0 max 16384 2023-08-25T04:59:12.352 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:59:12.352 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:59:12.352 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:59:12.352 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:59:12.353 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:59:12.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 msgr2=0x7fa1c00a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 0x7fa1c00a2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0094110 tx=0x7fa1d0007f10 comp rx=0 tx=0).stop 2023-08-25T04:59:12.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 msgr2=0x7fa1d8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:12.355 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007ee0 tx=0x7fa1d000d3b0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 shutdown_connections 2023-08-25T04:59:12.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d8108aa0 0x7fa1d8198a10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fa1c00a0140 0x7fa1c00a2600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1d8102a90 0x7fa1d819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.356 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 --2- 172.21.15.145:0/2517720129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d8103450 0x7fa1d819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:12.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 >> 172.21.15.145:0/2517720129 conn(0x7fa1d80fe580 msgr2=0x7fa1d80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:12.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 shutdown_connections 2023-08-25T04:59:12.357 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:11.284+0000 7fa1ddfff700 1 -- 172.21.15.145:0/2517720129 wait complete. 2023-08-25T04:59:12.358 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:59:12.358 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:11 smithi145 ceph-mon[97681]: pgmap v440: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T04:59:12.359 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:11 smithi145 ceph-mon[97681]: osdmap e105: 12 total, 12 up, 12 in 2023-08-25T04:59:14.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:13 smithi162 ceph-mon[130941]: pgmap v442: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:14.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:13 smithi183 ceph-mon[131414]: pgmap v442: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:13 smithi145 ceph-mon[97681]: pgmap v442: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:15 smithi162 ceph-mon[130941]: pgmap v443: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:16.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:15 smithi183 ceph-mon[131414]: pgmap v443: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:16.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:15 smithi145 ceph-mon[97681]: pgmap v443: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T04:59:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:17 smithi162 ceph-mon[130941]: pgmap v444: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:17 smithi183 ceph-mon[131414]: pgmap v444: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:18.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:17 smithi145 ceph-mon[97681]: pgmap v444: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:19 smithi162 ceph-mon[130941]: pgmap v445: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:19 smithi162 ceph-mon[130941]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T04:59:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:19 smithi183 ceph-mon[131414]: pgmap v445: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:20.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:19 smithi183 ceph-mon[131414]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T04:59:20.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:19 smithi145 ceph-mon[97681]: pgmap v445: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:20.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:19 smithi145 ceph-mon[97681]: osdmap e106: 12 total, 12 up, 12 in 2023-08-25T04:59:22.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:21 smithi162 ceph-mon[130941]: pgmap v447: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2023-08-25T04:59:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:21 smithi183 ceph-mon[131414]: pgmap v447: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2023-08-25T04:59:22.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:21 smithi145 ceph-mon[97681]: pgmap v447: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2023-08-25T04:59:24.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:23 smithi162 ceph-mon[130941]: pgmap v448: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:24.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:23 smithi183 ceph-mon[131414]: pgmap v448: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:23 smithi145 ceph-mon[97681]: pgmap v448: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:25 smithi162 ceph-mon[130941]: pgmap v449: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:26.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:25 smithi183 ceph-mon[131414]: pgmap v449: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:26.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:25 smithi145 ceph-mon[97681]: pgmap v449: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T04:59:28.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:27 smithi162 ceph-mon[130941]: pgmap v450: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:27 smithi183 ceph-mon[131414]: pgmap v450: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:28.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:27 smithi145 ceph-mon[97681]: pgmap v450: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:30.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:29 smithi183 ceph-mon[131414]: pgmap v451: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:29 smithi145 ceph-mon[97681]: pgmap v451: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:30.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:29 smithi162 ceph-mon[130941]: pgmap v451: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T04:59:32.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:31 smithi183 ceph-mon[131414]: pgmap v452: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:59:32.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:31 smithi145 ceph-mon[97681]: pgmap v452: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:59:32.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:31 smithi162 ceph-mon[130941]: pgmap v452: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 94 B/s wr, 8 op/s 2023-08-25T04:59:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:33 smithi183 ceph-mon[131414]: pgmap v453: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:34.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:33 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:59:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:33 smithi145 ceph-mon[97681]: pgmap v453: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:34.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:33 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:59:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:33 smithi162 ceph-mon[130941]: pgmap v453: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:33 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T04:59:35.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:34 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:59:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:34 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:59:35.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:34 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:59:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:34 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:59:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:34 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:59:35.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:34 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:59:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:34 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T04:59:35.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:34 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T04:59:35.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:34 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T04:59:36.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:35 smithi183 ceph-mon[131414]: pgmap v454: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:35 smithi145 ceph-mon[97681]: pgmap v454: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:36.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:35 smithi162 ceph-mon[130941]: pgmap v454: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:38.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:37 smithi145 ceph-mon[97681]: pgmap v455: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:38.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:37 smithi162 ceph-mon[130941]: pgmap v455: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:38.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:37 smithi183 ceph-mon[131414]: pgmap v455: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:40.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:39 smithi145 ceph-mon[97681]: pgmap v456: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:40.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:39 smithi162 ceph-mon[130941]: pgmap v456: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:40.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:39 smithi183 ceph-mon[131414]: pgmap v456: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:42.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:41 smithi145 ceph-mon[97681]: pgmap v457: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:42.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T04:59:42.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:41 smithi162 ceph-mon[130941]: pgmap v457: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:42.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:41 smithi183 ceph-mon[131414]: pgmap v457: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:42.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 -- 172.21.15.145:0/2857362842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 msgr2=0x7fc0e810d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 --2- 172.21.15.145:0/2857362842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e810d740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000b0a0 tx=0x7fc0e000b3b0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 -- 172.21.15.145:0/2857362842 shutdown_connections 2023-08-25T04:59:42.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 --2- 172.21.15.145:0/2857362842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e810d740 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 --2- 172.21.15.145:0/2857362842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 0x7fc0e8106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 --2- 172.21.15.145:0/2857362842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 0x7fc0e8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 -- 172.21.15.145:0/2857362842 >> 172.21.15.145:0/2857362842 conn(0x7fc0e80fc3a0 msgr2=0x7fc0e80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 -- 172.21.15.145:0/2857362842 shutdown_connections 2023-08-25T04:59:42.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.587+0000 7fc0eebff700 1 -- 172.21.15.145:0/2857362842 wait complete. 2023-08-25T04:59:42.593 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.588+0000 7fc0eebff700 1 Processor -- start 2023-08-25T04:59:42.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.588+0000 7fc0eebff700 1 -- start start 2023-08-25T04:59:42.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 0x7fc0e8072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 0x7fc0e81a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e8073990 con 0x7fc0e8106880 2023-08-25T04:59:42.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a5040 con 0x7fc0e8100870 2023-08-25T04:59:42.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0eebff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e81a5180 con 0x7fc0e8101230 2023-08-25T04:59:42.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53104/0 (socket says 172.21.15.145:53104) 2023-08-25T04:59:42.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0e7fff700 1 -- 172.21.15.145:0/1172241055 learned_addr learned my addr 172.21.15.145:0/1172241055 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:42.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 0x7fc0e8072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.589+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 0x7fc0e81a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 -- 172.21.15.145:0/1172241055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 msgr2=0x7fc0e8072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 0x7fc0e8072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 -- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 msgr2=0x7fc0e81a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 0x7fc0e81a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 -- 172.21.15.145:0/1172241055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0009d00 con 0x7fc0e8101230 2023-08-25T04:59:42.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9a0 tx=0x7fc0d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e5ffb700 1 -- 172.21.15.145:0/1172241055 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4009790 con 0x7fc0e8101230 2023-08-25T04:59:42.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e5ffb700 1 -- 172.21.15.145:0/1172241055 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d4010430 con 0x7fc0e8101230 2023-08-25T04:59:42.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0e5ffb700 1 -- 172.21.15.145:0/1172241055 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4018680 con 0x7fc0e8101230 2023-08-25T04:59:42.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e81a53e0 con 0x7fc0e8101230 2023-08-25T04:59:42.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.590+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e81a5870 con 0x7fc0e8101230 2023-08-25T04:59:42.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.594+0000 7fc0e5ffb700 1 -- 172.21.15.145:0/1172241055 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fc0d40105a0 con 0x7fc0e8101230 2023-08-25T04:59:42.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.595+0000 7fc1025a8b80 1 Processor -- start 2023-08-25T04:59:42.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.595+0000 7fc1025a8b80 1 -- start start 2023-08-25T04:59:42.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048427b0 con 0x5578048e0380 2023-08-25T04:59:42.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557804833780 con 0x5578048dac40 2023-08-25T04:59:42.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048310a0 con 0x5578048e0ca0 2023-08-25T04:59:42.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53118/0 (socket says 172.21.15.145:53118) 2023-08-25T04:59:42.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 -- 172.21.15.145:0/3233900751 learned_addr learned my addr 172.21.15.145:0/3233900751 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:42.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0e5ffb700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 0x7fc0d80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 -- 172.21.15.145:0/3233900751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 msgr2=0x5578048db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.608 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 -- 172.21.15.145:0/3233900751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 msgr2=0x5578048e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ec99b700 1 -- 172.21.15.145:0/3233900751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578046ac630 con 0x5578048e0ca0 2023-08-25T04:59:42.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0e5ffb700 1 -- 172.21.15.145:0/1172241055 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc0d4014070 con 0x7fc0e8101230 2023-08-25T04:59:42.609 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T04:59:42.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.596+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 0x7fc0d80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.597+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009d00 tx=0x7fc0dc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132278512b888fb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.597+0000 7fc0d2ffd700 1 -- 172.21.15.145:0/3233900751 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc004230 con 0x5578048e0ca0 2023-08-25T04:59:42.610 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.597+0000 7fc0d2ffd700 1 -- 172.21.15.145:0/3233900751 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0dc004390 con 0x5578048e0ca0 2023-08-25T04:59:42.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.597+0000 7fc0d2ffd700 1 -- 172.21.15.145:0/3233900751 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc019880 con 0x5578048e0ca0 2023-08-25T04:59:42.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 -- 172.21.15.145:0/3233900751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 msgr2=0x5578048da700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 0x7fc0d80a2600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8074d30 tx=0x7fc0e00099e0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009d00 tx=0x7fc0dc011720 comp rx=0 tx=0).stop 2023-08-25T04:59:42.611 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 -- 172.21.15.145:0/3233900751 shutdown_connections 2023-08-25T04:59:42.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.612 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.616 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/3233900751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048da700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.617 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 -- 172.21.15.145:0/3233900751 >> 172.21.15.145:0/3233900751 conn(0x55780484f0e0 msgr2=0x5578048d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 -- 172.21.15.145:0/3233900751 shutdown_connections 2023-08-25T04:59:42.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.598+0000 7fc1025a8b80 1 -- 172.21.15.145:0/3233900751 wait complete. 2023-08-25T04:59:42.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.599+0000 7fc1025a8b80 1 -- start start 2023-08-25T04:59:42.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.599+0000 7fc1025a8b80 1 Processor -- start 2023-08-25T04:59:42.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780482c990 con 0x5578048e0380 2023-08-25T04:59:42.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578047cabb0 con 0x5578048dac40 2023-08-25T04:59:42.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578047ca8b0 con 0x5578048e0ca0 2023-08-25T04:59:42.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc1025a8b80 10 client.0 _ll_register_callbacks cb 0x557804714a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:59:42.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53126/0 (socket says 172.21.15.145:53126) 2023-08-25T04:59:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 -- 172.21.15.145:0/1372884055 learned_addr learned my addr 172.21.15.145:0/1372884055 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 -- 172.21.15.145:0/1372884055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 msgr2=0x5578048e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 -- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 msgr2=0x5578048e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.600+0000 7fc0ed19c700 1 -- 172.21.15.145:0/1372884055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc009960 con 0x5578048e0ca0 2023-08-25T04:59:42.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0015000 tx=0x7fc0e0006230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8bdc5d9031c066a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:59:42.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e000e040 con 0x5578048e0ca0 2023-08-25T04:59:42.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578048aa1c0 con 0x5578048e0ca0 2023-08-25T04:59:42.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc0e0005450 con 0x5578048e0ca0 2023-08-25T04:59:42.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e0005610 con 0x5578048e0ca0 2023-08-25T04:59:42.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc1025a8b80 10 client.26278 fetch_fsmap learned FSMap version 35 2023-08-25T04:59:42.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557804658de0 con 0x5578048e0ca0 2023-08-25T04:59:42.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.601+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578048aa1c0 con 0x5578048e0ca0 2023-08-25T04:59:42.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0e0025070 con 0x5578048e0ca0 2023-08-25T04:59:42.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 10 client.26278 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:59:42.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:59:42.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 20 client.26278 populate_metadata read hostname 'smithi145' 2023-08-25T04:59:42.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5578048f4720 0x5578048f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 4 client.26278 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T04:59:42.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578047d2ba0 con 0x5578048f4720 2023-08-25T04:59:42.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.602+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5578048f4720 0x5578048f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.603+0000 7fc0c7fff700 10 client.26278 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:42.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.603+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5578048f4720 0x5578048f6b40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7cf74d58ad7afc server_cookie=9c7343e7b513c516 in_seq=0 out_seq=0 2023-08-25T04:59:42.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.603+0000 7fc0c7fff700 1 -- 172.21.15.145:0/1372884055 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5578047d2ba0 con 0x5578048f4720 2023-08-25T04:59:42.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.603+0000 7fc0c7fff700 10 client.26278 handle_command_reply: tid=0 2023-08-25T04:59:42.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.603+0000 7fc1025a8b80 1 client.26278 shutdown 2023-08-25T04:59:42.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 msgr2=0x5578048e4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0015000 tx=0x7fc0e0006230 comp rx=0 tx=0).stop 2023-08-25T04:59:42.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 shutdown_connections 2023-08-25T04:59:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x5578048f4720 0x5578048f6b40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0ca0 0x5578048e4580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/1372884055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0380 0x5578048e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc0c7fff700 0 client.26278 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:42.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 >> 172.21.15.145:0/1372884055 conn(0x55780484f0e0 msgr2=0x5578048cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 shutdown_connections 2023-08-25T04:59:42.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 1 -- 172.21.15.145:0/1372884055 wait complete. 2023-08-25T04:59:42.632 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.604+0000 7fc1025a8b80 20 client.26278 trim_cache size 0 max 16384 2023-08-25T04:59:42.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.630+0000 7fc1025a8b80 1 Processor -- start 2023-08-25T04:59:42.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.630+0000 7fc1025a8b80 1 -- start start 2023-08-25T04:59:42.634 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0ca0 0x5578048de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578046ae920 con 0x5578048e0ca0 2023-08-25T04:59:42.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048e4930 con 0x5578048dac40 2023-08-25T04:59:42.635 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048f4500 con 0x5578048e0380 2023-08-25T04:59:42.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0ec99b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.636 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53136/0 (socket says 172.21.15.145:53136) 2023-08-25T04:59:42.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2863301405 learned_addr learned my addr 172.21.15.145:0/2863301405 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:42.637 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0ca0 0x5578048de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2863301405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 msgr2=0x5578048da700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048da700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.638 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2863301405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0ca0 msgr2=0x5578048de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0ca0 0x5578048de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.639 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.631+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2863301405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557804641c60 con 0x5578048e0380 2023-08-25T04:59:42.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.632+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4095910 tx=0x7fc0d406bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f23abfdc152cd432 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.632+0000 7fc0d37fe700 1 -- 172.21.15.145:0/2863301405 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4098ea0 con 0x5578048e0380 2023-08-25T04:59:42.640 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.632+0000 7fc0d37fe700 1 -- 172.21.15.145:0/2863301405 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d40b6e90 con 0x5578048e0380 2023-08-25T04:59:42.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.632+0000 7fc0d37fe700 1 -- 172.21.15.145:0/2863301405 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40649a0 con 0x5578048e0380 2023-08-25T04:59:42.641 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2863301405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 msgr2=0x5578048eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4095910 tx=0x7fc0d406bfc0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2863301405 shutdown_connections 2023-08-25T04:59:42.642 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048e0ca0 0x5578048de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048dac40 0x5578048da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2863301405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb7d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2863301405 >> 172.21.15.145:0/2863301405 conn(0x55780484f0e0 msgr2=0x5578048d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.643 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2863301405 shutdown_connections 2023-08-25T04:59:42.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2863301405 wait complete. 2023-08-25T04:59:42.644 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.633+0000 7fc1025a8b80 1 -- start start 2023-08-25T04:59:42.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 Processor -- start 2023-08-25T04:59:42.648 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 0x5578048eb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.649 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048e0ca0 0x5578048e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048e9ec0 con 0x5578048dac40 2023-08-25T04:59:42.650 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048ea030 con 0x5578048e0ca0 2023-08-25T04:59:42.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.634+0000 7fc1025a8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578048ea1f0 con 0x5578048e0380 2023-08-25T04:59:42.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc1025a8b80 10 client.0 _ll_register_callbacks cb 0x7fc0e0003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T04:59:42.651 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0ec99b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 0x5578048eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.652 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:53144/0 (socket says 172.21.15.145:53144) 2023-08-25T04:59:42.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2909534927 learned_addr learned my addr 172.21.15.145:0/2909534927 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T04:59:42.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0ed19c700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048e0ca0 0x5578048e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.653 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2909534927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048e0ca0 msgr2=0x5578048e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048e0ca0 0x5578048e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 msgr2=0x5578048eb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T04:59:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 0x5578048eb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 -- 172.21.15.145:0/2909534927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e81a5870 con 0x5578048e0380 2023-08-25T04:59:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 0x5578048eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T04:59:42.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0e7fff700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c2d0 tx=0x7fc0d4099660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3131b82ca99eb81f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T04:59:42.656 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T04:59:42.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.635+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4099300 con 0x5578048e0380 2023-08-25T04:59:42.657 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc0d40b1020 con 0x5578048e0380 2023-08-25T04:59:42.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578048cd8b0 con 0x5578048e0380 2023-08-25T04:59:42.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0d40a0cb0 con 0x5578048e0380 2023-08-25T04:59:42.658 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 10 client.26284 fetch_fsmap learned FSMap version 35 2023-08-25T04:59:42.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55780467f250 con 0x5578048e0380 2023-08-25T04:59:42.659 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578048cd8b0 con 0x5578048e0380 2023-08-25T04:59:42.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0d40a0900 con 0x5578048e0380 2023-08-25T04:59:42.660 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T04:59:42.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T04:59:42.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 20 client.26284 populate_metadata read hostname 'smithi145' 2023-08-25T04:59:42.661 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557804949360 0x55780494b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T04:59:42.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 4 client.26284 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T04:59:42.662 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.636+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557804825970 con 0x557804949360 2023-08-25T04:59:42.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557804949360 0x55780494b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T04:59:42.663 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc0ec99b700 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557804949360 0x55780494b780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e336004b4fc631e0 server_cookie=90e3a1f6ddb09690 in_seq=0 out_seq=0 2023-08-25T04:59:42.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc0c7fff700 10 client.26284 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:42.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc0c7fff700 1 -- 172.21.15.145:0/2909534927 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557804825970 con 0x557804949360 2023-08-25T04:59:42.664 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc0c7fff700 10 client.26284 handle_command_reply: tid=0 2023-08-25T04:59:42.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc1025a8b80 1 client.26284 shutdown 2023-08-25T04:59:42.665 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 msgr2=0x5578048eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.637+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c2d0 tx=0x7fc0d4099660 comp rx=0 tx=0).stop 2023-08-25T04:59:42.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 shutdown_connections 2023-08-25T04:59:42.666 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5578048e0ca0 0x5578048e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578048dac40 0x5578048eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.667 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x557804949360 0x55780494b780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 --2- 172.21.15.145:0/2909534927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578048e0380 0x5578048eb750 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.668 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc0c7fff700 0 client.26284 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T04:59:42.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 >> 172.21.15.145:0/2909534927 conn(0x55780484f0e0 msgr2=0x5578048de050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 shutdown_connections 2023-08-25T04:59:42.669 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 1 -- 172.21.15.145:0/2909534927 wait complete. 2023-08-25T04:59:42.670 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc1025a8b80 20 client.26284 trim_cache size 0 max 16384 2023-08-25T04:59:42.670 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T04:59:42.671 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T04:59:42.671 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T04:59:42.671 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T04:59:42.672 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T04:59:42.672 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T04:59:42.672 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T04:59:42.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:59:42.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T04:59:42.673 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T04:59:42.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.638+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 msgr2=0x7fc0d80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.675 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 0x7fc0d80a2600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8074d30 tx=0x7fc0e00099e0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 msgr2=0x7fc0e8073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T04:59:42.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9a0 tx=0x7fc0d400ecb0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.676 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 shutdown_connections 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0e8106880 0x7fc0e81a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc0d80a0140 0x7fc0d80a2600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e8100870 0x7fc0e8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 --2- 172.21.15.145:0/1172241055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0e8101230 0x7fc0e8073160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 >> 172.21.15.145:0/1172241055 conn(0x7fc0e80fc3a0 msgr2=0x7fc0e80fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T04:59:42.677 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 shutdown_connections 2023-08-25T04:59:42.678 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T04:59:42.639+0000 7fc0eebff700 1 -- 172.21.15.145:0/1172241055 wait complete. 2023-08-25T04:59:42.678 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T04:59:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:43 smithi145 ceph-mon[97681]: pgmap v458: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:44.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:43 smithi162 ceph-mon[130941]: pgmap v458: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:44.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:43 smithi183 ceph-mon[131414]: pgmap v458: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:46.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:45 smithi145 ceph-mon[97681]: pgmap v459: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:46.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:45 smithi162 ceph-mon[130941]: pgmap v459: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:46.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:45 smithi183 ceph-mon[131414]: pgmap v459: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:48.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:47 smithi145 ceph-mon[97681]: pgmap v460: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:48.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:47 smithi162 ceph-mon[130941]: pgmap v460: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:48.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:47 smithi183 ceph-mon[131414]: pgmap v460: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:50.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:49 smithi145 ceph-mon[97681]: pgmap v461: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:50.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:49 smithi162 ceph-mon[130941]: pgmap v461: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:50.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:49 smithi183 ceph-mon[131414]: pgmap v461: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:52.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:51 smithi145 ceph-mon[97681]: pgmap v462: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:52.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:51 smithi162 ceph-mon[130941]: pgmap v462: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:52.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:51 smithi183 ceph-mon[131414]: pgmap v462: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:54.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:53 smithi145 ceph-mon[97681]: pgmap v463: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:53 smithi162 ceph-mon[130941]: pgmap v463: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:54.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:53 smithi183 ceph-mon[131414]: pgmap v463: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:55 smithi145 ceph-mon[97681]: pgmap v464: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:56.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:55 smithi162 ceph-mon[130941]: pgmap v464: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:56.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:55 smithi183 ceph-mon[131414]: pgmap v464: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T04:59:58.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:57 smithi145 ceph-mon[97681]: pgmap v465: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:57 smithi162 ceph-mon[130941]: pgmap v465: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T04:59:58.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:57 smithi183 ceph-mon[131414]: pgmap v465: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:00.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 04:59:59 smithi145 ceph-mon[97681]: pgmap v466: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:00.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 04:59:59 smithi162 ceph-mon[130941]: pgmap v466: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:00.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 04:59:59 smithi183 ceph-mon[131414]: pgmap v466: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:01.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:00 smithi145 ceph-mon[97681]: overall HEALTH_OK 2023-08-25T05:00:01.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:00 smithi162 ceph-mon[130941]: overall HEALTH_OK 2023-08-25T05:00:01.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:00 smithi183 ceph-mon[131414]: overall HEALTH_OK 2023-08-25T05:00:02.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:01 smithi145 ceph-mon[97681]: pgmap v467: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:00:02.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:01 smithi162 ceph-mon[130941]: pgmap v467: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:00:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:01 smithi183 ceph-mon[131414]: pgmap v467: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:00:04.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:03 smithi145 ceph-mon[97681]: pgmap v468: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:04.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:03 smithi162 ceph-mon[130941]: pgmap v468: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:03 smithi183 ceph-mon[131414]: pgmap v468: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:06.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:05 smithi145 ceph-mon[97681]: pgmap v469: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:06.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:05 smithi162 ceph-mon[130941]: pgmap v469: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:05 smithi183 ceph-mon[131414]: pgmap v469: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:00:07.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:06 smithi145 ceph-mon[97681]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T05:00:07.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:06 smithi162 ceph-mon[130941]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T05:00:07.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:06 smithi183 ceph-mon[131414]: osdmap e107: 12 total, 12 up, 12 in 2023-08-25T05:00:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:07 smithi145 ceph-mon[97681]: pgmap v471: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:08.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:07 smithi162 ceph-mon[130941]: pgmap v471: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:07 smithi183 ceph-mon[131414]: pgmap v471: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:10.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:09 smithi145 ceph-mon[97681]: pgmap v472: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:10.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:09 smithi162 ceph-mon[130941]: pgmap v472: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:10.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:09 smithi183 ceph-mon[131414]: pgmap v472: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:00:12.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:11 smithi145 ceph-mon[97681]: pgmap v473: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:12.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:11 smithi162 ceph-mon[130941]: pgmap v473: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:12.423 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:11 smithi183 ceph-mon[131414]: pgmap v473: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:12.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T05:00:12.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 -- 172.21.15.145:0/1865347606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb664101a30 msgr2=0x7fb664101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.936 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/1865347606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb664101a30 0x7fb664101e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb660009990 tx=0x7fb660009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 -- 172.21.15.145:0/1865347606 shutdown_connections 2023-08-25T05:00:12.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/1865347606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb66410ae50 0x7fb66410d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/1865347606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb664101a30 0x7fb664101e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/1865347606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664102350 0x7fb66410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 -- 172.21.15.145:0/1865347606 >> 172.21.15.145:0/1865347606 conn(0x7fb6640fb270 msgr2=0x7fb6640fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:12.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 -- 172.21.15.145:0/1865347606 shutdown_connections 2023-08-25T05:00:12.938 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.935+0000 7fb66b3ff700 1 -- 172.21.15.145:0/1865347606 wait complete. 2023-08-25T05:00:12.939 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.936+0000 7fb66b3ff700 1 Processor -- start 2023-08-25T05:00:12.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 -- start start 2023-08-25T05:00:12.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 0x7fb6640ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb664102350 0x7fb6640ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.940 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 0x7fb6641a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641003f0 con 0x7fb664101a30 2023-08-25T05:00:12.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a50f0 con 0x7fb664102350 2023-08-25T05:00:12.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66b3ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641a5230 con 0x7fb66410ae50 2023-08-25T05:00:12.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 0x7fb6640ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.941 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 0x7fb6641a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 0x7fb6640ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41862/0 (socket says 172.21.15.145:41862) 2023-08-25T05:00:12.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.937+0000 7fb66919b700 1 -- 172.21.15.145:0/555960722 learned_addr learned my addr 172.21.15.145:0/555960722 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:12.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66899a700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb664102350 0x7fb6640ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 -- 172.21.15.145:0/555960722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb664102350 msgr2=0x7fb6640ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.942 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb664102350 0x7fb6640ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 -- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 msgr2=0x7fb6640ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 0x7fb6640ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 -- 172.21.15.145:0/555960722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600095f0 con 0x7fb66410ae50 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66999c700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 0x7fb6641a4bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00efc0 tx=0x7fb65c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb65a7fc700 1 -- 172.21.15.145:0/555960722 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c018070 con 0x7fb66410ae50 2023-08-25T05:00:12.943 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb65a7fc700 1 -- 172.21.15.145:0/555960722 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c00f040 con 0x7fb66410ae50 2023-08-25T05:00:12.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6641a5430 con 0x7fb66410ae50 2023-08-25T05:00:12.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.938+0000 7fb65a7fc700 1 -- 172.21.15.145:0/555960722 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00cd40 con 0x7fb66410ae50 2023-08-25T05:00:12.944 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.939+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a5910 con 0x7fb66410ae50 2023-08-25T05:00:12.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.942+0000 7fb65a7fc700 1 -- 172.21.15.145:0/555960722 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fb65c003a00 con 0x7fb66410ae50 2023-08-25T05:00:12.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.944+0000 7fb65a7fc700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 0x7fb6500a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.945 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.944+0000 7fb65a7fc700 1 -- 172.21.15.145:0/555960722 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fb65c0cb4f0 con 0x7fb66410ae50 2023-08-25T05:00:12.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.944+0000 7fb66919b700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 0x7fb6500a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.944+0000 7fb67ed91b80 1 Processor -- start 2023-08-25T05:00:12.946 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 -- start start 2023-08-25T05:00:12.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66919b700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 0x7fb6500a2600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb66000b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.948 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95b9c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95b047b0 con 0x556c95ba2ca0 2023-08-25T05:00:12.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95af5780 con 0x556c95ba2380 2023-08-25T05:00:12.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95af30a0 con 0x556c95b9cc40 2023-08-25T05:00:12.949 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66899a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95b9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66899a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95b9c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41870/0 (socket says 172.21.15.145:41870) 2023-08-25T05:00:12.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66899a700 1 -- 172.21.15.145:0/2793573782 learned_addr learned my addr 172.21.15.145:0/2793573782 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:12.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.945+0000 7fb66919b700 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.950 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 -- 172.21.15.145:0/2793573782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 msgr2=0x556c95ba2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 -- 172.21.15.145:0/2793573782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 msgr2=0x556c95b9c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95b9c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 -- 172.21.15.145:0/2793573782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c9596e630 con 0x556c95b9cc40 2023-08-25T05:00:12.951 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb66999c700 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9d0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0c88b0 tx=0x7fb65c098440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abce9ec864fd3ce1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb64f7fe700 1 -- 172.21.15.145:0/2793573782 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c01b320 con 0x556c95b9cc40 2023-08-25T05:00:12.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb64f7fe700 1 -- 172.21.15.145:0/2793573782 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c099000 con 0x556c95b9cc40 2023-08-25T05:00:12.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.946+0000 7fb64f7fe700 1 -- 172.21.15.145:0/2793573782 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0cb150 con 0x556c95b9cc40 2023-08-25T05:00:12.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2793573782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 msgr2=0x556c95b9d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.952 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9d0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0c88b0 tx=0x7fb65c098440 comp rx=0 tx=0).stop 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2793573782 shutdown_connections 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9d0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2793573782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95b9c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2793573782 >> 172.21.15.145:0/2793573782 conn(0x556c95b110e0 msgr2=0x556c95b97410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:12.953 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2793573782 shutdown_connections 2023-08-25T05:00:12.954 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.947+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2793573782 wait complete. 2023-08-25T05:00:12.956 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 -- start start 2023-08-25T05:00:12.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 Processor -- start 2023-08-25T05:00:12.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 0x556c95ba5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2ca0 0x556c95ba6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.957 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95aee990 con 0x556c95b9cc40 2023-08-25T05:00:12.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95a8cbb0 con 0x556c95ba2ca0 2023-08-25T05:00:12.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.948+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95a8c8b0 con 0x556c95ba2380 2023-08-25T05:00:12.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb67ed91b80 10 client.0 _ll_register_callbacks cb 0x556c959d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T05:00:12.958 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:47130/0 (socket says 172.21.15.145:47130) 2023-08-25T05:00:12.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 -- 172.21.15.145:0/2292770143 learned_addr learned my addr 172.21.15.145:0/2292770143 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:12.959 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66999c700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2ca0 0x556c95ba6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66919b700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 0x556c95ba5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 -- 172.21.15.145:0/2292770143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2ca0 msgr2=0x556c95ba6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.962 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2ca0 0x556c95ba6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 -- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 msgr2=0x556c95ba5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 0x556c95ba5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 -- 172.21.15.145:0/2292770143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641a5910 con 0x556c95ba2380 2023-08-25T05:00:12.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66919b700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 0x556c95ba5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:00:12.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb66899a700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb654012840 tx=0x7fb654012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866e2958bb029284 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb64cff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T05:00:12.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.949+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654017070 con 0x556c95ba2380 2023-08-25T05:00:12.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c95b6c1c0 con 0x556c95ba2380 2023-08-25T05:00:12.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fb654004050 con 0x556c95ba2380 2023-08-25T05:00:12.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb654004210 con 0x556c95ba2380 2023-08-25T05:00:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 10 client.26296 fetch_fsmap learned FSMap version 35 2023-08-25T05:00:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556c9591ade0 con 0x556c95ba2380 2023-08-25T05:00:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c95b6c1c0 con 0x556c95ba2380 2023-08-25T05:00:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb65400f5f0 con 0x556c95ba2380 2023-08-25T05:00:12.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.950+0000 7fb67ed91b80 20 client.26296 populate_metadata read hostname 'smithi145' 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95bb6890 0x556c95bb8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb67ed91b80 4 client.26296 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c95a94ba0 con 0x556c95bb6890 2023-08-25T05:00:12.967 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb66919b700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95bb6890 0x556c95bb8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb66919b700 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95bb6890 0x556c95bb8cb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476d27198092f8a server_cookie=a954bde74faed4a7 in_seq=0 out_seq=0 2023-08-25T05:00:12.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.951+0000 7fb64cff9700 10 client.26296 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:12.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb64cff9700 1 -- 172.21.15.145:0/2292770143 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c95a94ba0 con 0x556c95bb6890 2023-08-25T05:00:12.968 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb64cff9700 10 client.26296 handle_command_reply: tid=0 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 client.26296 shutdown 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 msgr2=0x556c95ba64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb654012840 tx=0x7fb654012b50 comp rx=0 tx=0).stop 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 shutdown_connections 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2ca0 0x556c95ba6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.969 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95b9cc40 0x556c95ba5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95bb6890 0x556c95bb8cb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/2292770143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2380 0x556c95ba64e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb64cff9700 0 client.26296 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:12.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.952+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 >> 172.21.15.145:0/2292770143 conn(0x556c95b110e0 msgr2=0x556c95b91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:12.970 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.953+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 shutdown_connections 2023-08-25T05:00:12.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.953+0000 7fb67ed91b80 1 -- 172.21.15.145:0/2292770143 wait complete. 2023-08-25T05:00:12.971 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.953+0000 7fb67ed91b80 20 client.26296 trim_cache size 0 max 16384 2023-08-25T05:00:12.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 Processor -- start 2023-08-25T05:00:12.973 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 -- start start 2023-08-25T05:00:12.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95bad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.974 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95970920 con 0x556c95ba2ca0 2023-08-25T05:00:12.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95b6c1c0 con 0x556c95ba2380 2023-08-25T05:00:12.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.970+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95ba5dd0 con 0x556c95b9cc40 2023-08-25T05:00:12.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:47136/0 (socket says 172.21.15.145:47136) 2023-08-25T05:00:12.975 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 -- 172.21.15.145:0/3821536926 learned_addr learned my addr 172.21.15.145:0/3821536926 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:12.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66999c700 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95bad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66899a700 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.976 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 -- 172.21.15.145:0/3821536926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 msgr2=0x556c95ba5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 -- 172.21.15.145:0/3821536926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 msgr2=0x556c95bad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95bad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 -- 172.21.15.145:0/3821536926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c95903c60 con 0x556c95b9cc40 2023-08-25T05:00:12.977 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb66919b700 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb660005910 tx=0x7fb660023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13afb84b9c1a3e2b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.971+0000 7fb64ffff700 1 -- 172.21.15.145:0/3821536926 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660005520 con 0x556c95b9cc40 2023-08-25T05:00:12.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb64ffff700 1 -- 172.21.15.145:0/3821536926 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb66002c050 con 0x556c95b9cc40 2023-08-25T05:00:12.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb64ffff700 1 -- 172.21.15.145:0/3821536926 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660004b10 con 0x556c95b9cc40 2023-08-25T05:00:12.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 -- 172.21.15.145:0/3821536926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 msgr2=0x556c95b9c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.978 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb660005910 tx=0x7fb660023000 comp rx=0 tx=0).stop 2023-08-25T05:00:12.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 -- 172.21.15.145:0/3821536926 shutdown_connections 2023-08-25T05:00:12.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2ca0 0x556c95bad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95b9cc40 0x556c95b9c700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/3821536926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95ba2380 0x556c95ba5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.979 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 -- 172.21.15.145:0/3821536926 >> 172.21.15.145:0/3821536926 conn(0x556c95b110e0 msgr2=0x556c95b91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:12.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.972+0000 7fb67ed91b80 1 -- 172.21.15.145:0/3821536926 shutdown_connections 2023-08-25T05:00:12.980 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.973+0000 7fb67ed91b80 1 -- 172.21.15.145:0/3821536926 wait complete. 2023-08-25T05:00:12.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.973+0000 7fb67ed91b80 1 -- start start 2023-08-25T05:00:12.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.973+0000 7fb67ed91b80 1 Processor -- start 2023-08-25T05:00:12.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95b9cc40 0x556c95b9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 0x556c95b9ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.983 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95b9f620 con 0x556c95ba2380 2023-08-25T05:00:12.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95c0ac00 con 0x556c95b9cc40 2023-08-25T05:00:12.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c95c0ad70 con 0x556c95ba2ca0 2023-08-25T05:00:12.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.974+0000 7fb67ed91b80 10 client.0 _ll_register_callbacks cb 0x7fb654009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T05:00:12.984 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66899a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 0x556c95b9ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:47142/0 (socket says 172.21.15.145:47142) 2023-08-25T05:00:12.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 -- 172.21.15.145:0/731017422 learned_addr learned my addr 172.21.15.145:0/731017422 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:12.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66919b700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95b9cc40 0x556c95b9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.985 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 -- 172.21.15.145:0/731017422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95b9cc40 msgr2=0x556c95b9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95b9cc40 0x556c95b9e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 -- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 msgr2=0x556c95b9ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 0x556c95b9ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 -- 172.21.15.145:0/731017422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66410dfd0 con 0x556c95ba2ca0 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66899a700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 0x556c95b9ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:00:12.986 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb66999c700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0c8960 tx=0x7fb65c096eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca0dd7627d3b09b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:12.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb64cff9700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T05:00:12.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c01b320 con 0x556c95ba2ca0 2023-08-25T05:00:12.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c95b8f8b0 con 0x556c95ba2ca0 2023-08-25T05:00:12.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.975+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9433+0+0 (secure 0 0 0) 0x7fb65c098900 con 0x556c95ba2ca0 2023-08-25T05:00:12.987 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb65c0cb360 con 0x556c95ba2ca0 2023-08-25T05:00:12.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 10 client.26302 fetch_fsmap learned FSMap version 35 2023-08-25T05:00:12.988 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556c95941250 con 0x556c95ba2ca0 2023-08-25T05:00:12.992 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c0b4020 con 0x556c95ba2ca0 2023-08-25T05:00:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb65c0a01f0 con 0x556c95ba2ca0 2023-08-25T05:00:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 10 client.26302 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T05:00:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T05:00:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 20 client.26302 populate_metadata read hostname 'smithi145' 2023-08-25T05:00:12.993 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95c0bfe0 0x556c95c0e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 4 client.26302 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T05:00:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c95ae7970 con 0x556c95c0bfe0 2023-08-25T05:00:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.976+0000 7fb66919b700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95c0bfe0 0x556c95c0e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb66919b700 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95c0bfe0 0x556c95c0e3d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa50318e448cb8ef server_cookie=9b5e6a6fb9c47f4a in_seq=0 out_seq=0 2023-08-25T05:00:12.994 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb64cff9700 10 client.26302 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb64cff9700 1 -- 172.21.15.145:0/731017422 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556c95ae7970 con 0x556c95c0bfe0 2023-08-25T05:00:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb64cff9700 10 client.26302 handle_command_reply: tid=0 2023-08-25T05:00:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 client.26302 shutdown 2023-08-25T05:00:12.995 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 msgr2=0x556c95c0a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0c8960 tx=0x7fb65c096eb0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 shutdown_connections 2023-08-25T05:00:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x556c95c0bfe0 0x556c95c0e3d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.977+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c95ba2ca0 0x556c95c0a6c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.996 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556c95b9cc40 0x556c95b9e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 1 --2- 172.21.15.145:0/731017422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556c95ba2380 0x556c95b9ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:12.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb64cff9700 0 client.26302 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:12.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 >> 172.21.15.145:0/731017422 conn(0x556c95b110e0 msgr2=0x556c95ba1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:12.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 shutdown_connections 2023-08-25T05:00:12.997 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 1 -- 172.21.15.145:0/731017422 wait complete. 2023-08-25T05:00:12.998 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb67ed91b80 20 client.26302 trim_cache size 0 max 16384 2023-08-25T05:00:12.999 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:12.999 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T05:00:12.999 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T05:00:13.000 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T05:00:13.000 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T05:00:13.000 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T05:00:13.001 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T05:00:13.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:13.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:13.001 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:00:13.002 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 msgr2=0x7fb6500a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 0x7fb6500a2600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb66000b260 comp rx=0 tx=0).stop 2023-08-25T05:00:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 msgr2=0x7fb6641a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:13.003 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 0x7fb6641a4bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00efc0 tx=0x7fb65c00c3a0 comp rx=0 tx=0).stop 2023-08-25T05:00:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 shutdown_connections 2023-08-25T05:00:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb66410ae50 0x7fb6641a4bb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb664101a30 0x7fb6640ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:13.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.978+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb6500a0140 0x7fb6500a2600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.979+0000 7fb66b3ff700 1 --2- 172.21.15.145:0/555960722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb664102350 0x7fb6640ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.979+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 >> 172.21.15.145:0/555960722 conn(0x7fb6640fb270 msgr2=0x7fb6640fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.979+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 shutdown_connections 2023-08-25T05:00:13.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:12.979+0000 7fb66b3ff700 1 -- 172.21.15.145:0/555960722 wait complete. 2023-08-25T05:00:13.006 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T05:00:14.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:13 smithi145 ceph-mon[97681]: pgmap v474: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:14.416 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:13 smithi162 ceph-mon[130941]: pgmap v474: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:14.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:13 smithi183 ceph-mon[131414]: pgmap v474: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:15 smithi162 ceph-mon[130941]: pgmap v475: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:15 smithi162 ceph-mon[130941]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T05:00:16.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:15 smithi183 ceph-mon[131414]: pgmap v475: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:16.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:15 smithi183 ceph-mon[131414]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T05:00:16.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:15 smithi145 ceph-mon[97681]: pgmap v475: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:16.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:15 smithi145 ceph-mon[97681]: osdmap e108: 12 total, 12 up, 12 in 2023-08-25T05:00:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:17 smithi162 ceph-mon[130941]: pgmap v477: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:17 smithi183 ceph-mon[131414]: pgmap v477: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:18.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:17 smithi145 ceph-mon[97681]: pgmap v477: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:20.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:19 smithi162 ceph-mon[130941]: pgmap v478: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:20.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:19 smithi183 ceph-mon[131414]: pgmap v478: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:20.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:19 smithi145 ceph-mon[97681]: pgmap v478: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:00:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:21 smithi162 ceph-mon[130941]: pgmap v479: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:21 smithi183 ceph-mon[131414]: pgmap v479: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:22.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:21 smithi145 ceph-mon[97681]: pgmap v479: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:23 smithi162 ceph-mon[130941]: pgmap v480: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:24.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:23 smithi183 ceph-mon[131414]: pgmap v480: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:24.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:23 smithi145 ceph-mon[97681]: pgmap v480: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:26.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:25 smithi162 ceph-mon[130941]: pgmap v481: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:25 smithi183 ceph-mon[131414]: pgmap v481: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:26.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:25 smithi145 ceph-mon[97681]: pgmap v481: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:00:28.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:27 smithi162 ceph-mon[130941]: pgmap v482: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T05:00:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:27 smithi183 ceph-mon[131414]: pgmap v482: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T05:00:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:27 smithi145 ceph-mon[97681]: pgmap v482: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2023-08-25T05:00:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:29 smithi162 ceph-mon[130941]: pgmap v483: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:29 smithi183 ceph-mon[131414]: pgmap v483: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:29 smithi145 ceph-mon[97681]: pgmap v483: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:32.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:31 smithi162 ceph-mon[130941]: pgmap v484: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:31 smithi183 ceph-mon[131414]: pgmap v484: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:32.334 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:31 smithi145 ceph-mon[97681]: pgmap v484: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:34.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:33 smithi162 ceph-mon[130941]: pgmap v485: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:34.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:33 smithi183 ceph-mon[131414]: pgmap v485: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:34.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:33 smithi145 ceph-mon[97681]: pgmap v485: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:35.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:34 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T05:00:35.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:34 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T05:00:35.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:34 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-25T05:00:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:35 smithi162 ceph-mon[130941]: pgmap v486: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:35 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T05:00:36.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:35 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T05:00:36.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:35 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T05:00:36.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:35 smithi183 ceph-mon[131414]: pgmap v486: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:36.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:35 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T05:00:36.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:35 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T05:00:36.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:35 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T05:00:36.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:35 smithi145 ceph-mon[97681]: pgmap v486: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:35 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-25T05:00:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:35 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-25T05:00:36.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:35 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' 2023-08-25T05:00:38.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:37 smithi162 ceph-mon[130941]: pgmap v487: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:38.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:37 smithi183 ceph-mon[131414]: pgmap v487: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:38.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:37 smithi145 ceph-mon[97681]: pgmap v487: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:40.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:39 smithi162 ceph-mon[130941]: pgmap v488: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:40.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:39 smithi183 ceph-mon[131414]: pgmap v488: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:40.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:39 smithi145 ceph-mon[97681]: pgmap v488: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:42.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:41 smithi162 ceph-mon[130941]: pgmap v489: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:42.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:41 smithi183 ceph-mon[131414]: pgmap v489: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:42.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:41 smithi145 ceph-mon[97681]: pgmap v489: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:43.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-25T05:00:43.243 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 -- 172.21.15.145:0/4116726650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8103300 msgr2=0x7fb0e810f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/4116726650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8103300 0x7fb0e810f810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009f90 tx=0x7fb0e000b330 comp rx=0 tx=0).stop 2023-08-25T05:00:43.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 -- 172.21.15.145:0/4116726650 shutdown_connections 2023-08-25T05:00:43.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/4116726650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8103300 0x7fb0e810f810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.244 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/4116726650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8108950 0x7fb0e8108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/4116726650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8102940 0x7fb0e8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 -- 172.21.15.145:0/4116726650 >> 172.21.15.145:0/4116726650 conn(0x7fb0e80fe3b0 msgr2=0x7fb0e81007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 -- 172.21.15.145:0/4116726650 shutdown_connections 2023-08-25T05:00:43.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.237+0000 7fb0ed39d700 1 -- 172.21.15.145:0/4116726650 wait complete. 2023-08-25T05:00:43.245 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 Processor -- start 2023-08-25T05:00:43.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 -- start start 2023-08-25T05:00:43.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8102940 0x7fb0e819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8103300 0x7fb0e819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.246 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8198d90 con 0x7fb0e8102940 2023-08-25T05:00:43.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8198f00 con 0x7fb0e8103300 2023-08-25T05:00:43.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ed39d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e81990c0 con 0x7fb0e8108950 2023-08-25T05:00:43.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.239+0000 7fb0ecb9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.247 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8102940 0x7fb0e819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35504/0 (socket says 172.21.15.145:35504) 2023-08-25T05:00:43.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3639696850 learned_addr learned my addr 172.21.15.145:0/3639696850 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8103300 0x7fb0e819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3639696850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8103300 msgr2=0x7fb0e819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.248 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8103300 0x7fb0e819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8102940 msgr2=0x7fb0e819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8102940 0x7fb0e819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3639696850 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009c40 con 0x7fb0e8108950 2023-08-25T05:00:43.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004a50 tx=0x7fb0e0004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.249 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0e57fa700 1 -- 172.21.15.145:0/3639696850 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000e040 con 0x7fb0e8108950 2023-08-25T05:00:43.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e8199340 con 0x7fb0e8108950 2023-08-25T05:00:43.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0e57fa700 1 -- 172.21.15.145:0/3639696850 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e000bbf0 con 0x7fb0e8108950 2023-08-25T05:00:43.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.240+0000 7fb0e57fa700 1 -- 172.21.15.145:0/3639696850 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00052a0 con 0x7fb0e8108950 2023-08-25T05:00:43.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.241+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e81a56a0 con 0x7fb0e8108950 2023-08-25T05:00:43.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.244+0000 7fb0e57fa700 1 -- 172.21.15.145:0/3639696850 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fb0e001d470 con 0x7fb0e8108950 2023-08-25T05:00:43.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.246+0000 7fb0e57fa700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 0x7fb0d00a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.246+0000 7fb101eb6b80 1 Processor -- start 2023-08-25T05:00:43.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.246+0000 7fb0e57fa700 1 -- 172.21.15.145:0/3639696850 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fb0e00ccf00 con 0x7fb0e8108950 2023-08-25T05:00:43.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.246+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 0x7fb0d00a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.246+0000 7fb101eb6b80 1 -- start start 2023-08-25T05:00:43.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b2f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b29700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a917b0 con 0x560e97b2f500 2023-08-25T05:00:43.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a82780 con 0x560e97b2fe20 2023-08-25T05:00:43.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a800a0 con 0x560e97b29c40 2023-08-25T05:00:43.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35516/0 (socket says 172.21.15.145:35516) 2023-08-25T05:00:43.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3530175725 learned_addr learned my addr 172.21.15.145:0/3530175725 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b2f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b29700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3530175725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 msgr2=0x560e97b29700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b29700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3530175725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 msgr2=0x560e97b2f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b2f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0ecb9c700 1 -- 172.21.15.145:0/3530175725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e978fb630 con 0x560e97b29c40 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.247+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b29700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:00:43.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.248+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 0x7fb0d00a2620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005fd0 tx=0x7fb0d8013840 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.248+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0096dd0 tx=0x7fb0e0096770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a4215d8a87d6b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.248+0000 7fb0ce7fc700 1 -- 172.21.15.145:0/3530175725 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009aa60 con 0x560e97b29c40 2023-08-25T05:00:43.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.248+0000 7fb0ce7fc700 1 -- 172.21.15.145:0/3530175725 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e000bd60 con 0x560e97b29c40 2023-08-25T05:00:43.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.248+0000 7fb0ce7fc700 1 -- 172.21.15.145:0/3530175725 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00ca590 con 0x560e97b29c40 2023-08-25T05:00:43.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- 172.21.15.145:0/3530175725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 msgr2=0x560e97b2a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0096dd0 tx=0x7fb0e0096770 comp rx=0 tx=0).stop 2023-08-25T05:00:43.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- 172.21.15.145:0/3530175725 shutdown_connections 2023-08-25T05:00:43.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b2a080 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b2f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/3530175725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b29700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- 172.21.15.145:0/3530175725 >> 172.21.15.145:0/3530175725 conn(0x560e97a9e0e0 msgr2=0x560e97b24940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- 172.21.15.145:0/3530175725 shutdown_connections 2023-08-25T05:00:43.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- 172.21.15.145:0/3530175725 wait complete. 2023-08-25T05:00:43.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.249+0000 7fb101eb6b80 1 -- start start 2023-08-25T05:00:43.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 Processor -- start 2023-08-25T05:00:43.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b29c40 0x560e97b35dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a7b990 con 0x560e97b29c40 2023-08-25T05:00:43.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a19bb0 con 0x560e97b2fe20 2023-08-25T05:00:43.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.250+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97a198b0 con 0x560e97b2f500 2023-08-25T05:00:43.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb101eb6b80 10 client.0 _ll_register_callbacks cb 0x560e97963a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T05:00:43.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35528/0 (socket says 172.21.15.145:35528) 2023-08-25T05:00:43.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2124827469 learned_addr learned my addr 172.21.15.145:0/2124827469 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b29c40 0x560e97b35dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2124827469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 msgr2=0x560e97b33610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b33610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.265 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b29c40 msgr2=0x560e97b35dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b29c40 0x560e97b35dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2124827469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e81a56a0 con 0x560e97b2f500 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.251+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc012700 tx=0x7fb0dc012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba425994cb07f051 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb0c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc017070 con 0x560e97b2f500 2023-08-25T05:00:43.266 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fb0dc004050 con 0x560e97b2f500 2023-08-25T05:00:43.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e97af91c0 con 0x560e97b2f500 2023-08-25T05:00:43.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0dc004210 con 0x560e97b2f500 2023-08-25T05:00:43.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb101eb6b80 10 client.26314 fetch_fsmap learned FSMap version 35 2023-08-25T05:00:43.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560e978a7de0 con 0x560e97b2f500 2023-08-25T05:00:43.267 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.252+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e97af91c0 con 0x560e97b2f500 2023-08-25T05:00:43.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb0dc00f510 con 0x560e97b2f500 2023-08-25T05:00:43.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 10 client.26314 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T05:00:43.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 10 client.26314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T05:00:43.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 20 client.26314 populate_metadata read hostname 'smithi145' 2023-08-25T05:00:43.268 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b436d0 0x560e97b45af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 4 client.26314 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T05:00:43.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e97a21ba0 con 0x560e97b436d0 2023-08-25T05:00:43.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.253+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b436d0 0x560e97b45af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.254+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b436d0 0x560e97b45af0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3e91fba2cc42a6 server_cookie=c08418833ebd74d4 in_seq=0 out_seq=0 2023-08-25T05:00:43.269 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.254+0000 7fb0c77fe700 10 client.26314 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:43.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.254+0000 7fb0c77fe700 1 -- 172.21.15.145:0/2124827469 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e97a21ba0 con 0x560e97b436d0 2023-08-25T05:00:43.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.254+0000 7fb0c77fe700 10 client.26314 handle_command_reply: tid=0 2023-08-25T05:00:43.270 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.254+0000 7fb101eb6b80 1 client.26314 shutdown 2023-08-25T05:00:43.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 msgr2=0x560e97b32fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.271 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc012700 tx=0x7fb0dc012a10 comp rx=0 tx=0).stop 2023-08-25T05:00:43.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 shutdown_connections 2023-08-25T05:00:43.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b29c40 0x560e97b35dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b436d0 0x560e97b45af0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.272 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2124827469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b32fe0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb0c77fe700 0 client.26314 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:43.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 >> 172.21.15.145:0/2124827469 conn(0x560e97a9e0e0 msgr2=0x560e97b286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 shutdown_connections 2023-08-25T05:00:43.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2124827469 wait complete. 2023-08-25T05:00:43.273 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.255+0000 7fb101eb6b80 20 client.26314 trim_cache size 0 max 16384 2023-08-25T05:00:43.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.278+0000 7fb101eb6b80 1 Processor -- start 2023-08-25T05:00:43.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 -- start start 2023-08-25T05:00:43.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 0x560e97b3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.282 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2fe20 0x560e97b2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e978fd920 con 0x560e97b2fe20 2023-08-25T05:00:43.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97b31560 con 0x560e97b29c40 2023-08-25T05:00:43.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97b1e930 con 0x560e97b2f500 2023-08-25T05:00:43.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 0x560e97b3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.283 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.284 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 0x560e97b3a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:49276/0 (socket says 172.21.15.145:49276) 2023-08-25T05:00:43.285 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35534/0 (socket says 172.21.15.145:35534) 2023-08-25T05:00:43.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.279+0000 7fb0e7fff700 1 -- 172.21.15.145:0/2170984491 learned_addr learned my addr 172.21.15.145:0/2170984491 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.286 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2fe20 0x560e97b2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2170984491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 msgr2=0x560e97b3a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.287 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 0x560e97b3a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2170984491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2fe20 msgr2=0x560e97b2cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.288 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2fe20 0x560e97b2cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 -- 172.21.15.145:0/2170984491 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e97890c60 con 0x560e97b2f500 2023-08-25T05:00:43.289 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00f870 tx=0x7fb0dc012320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c280c6939eda3923 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0ceffd700 1 -- 172.21.15.145:0/2170984491 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc017070 con 0x560e97b2f500 2023-08-25T05:00:43.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0ceffd700 1 -- 172.21.15.145:0/2170984491 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0dc0092d0 con 0x560e97b2f500 2023-08-25T05:00:43.290 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.280+0000 7fb0ceffd700 1 -- 172.21.15.145:0/2170984491 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00f5f0 con 0x560e97b2f500 2023-08-25T05:00:43.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2170984491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 msgr2=0x560e97b35900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00f870 tx=0x7fb0dc012320 comp rx=0 tx=0).stop 2023-08-25T05:00:43.291 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2170984491 shutdown_connections 2023-08-25T05:00:43.292 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2fe20 0x560e97b2cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b29c40 0x560e97b3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.293 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/2170984491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b2f500 0x560e97b35900 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2170984491 >> 172.21.15.145:0/2170984491 conn(0x560e97a9e0e0 msgr2=0x560e97b24530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2170984491 shutdown_connections 2023-08-25T05:00:43.294 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- 172.21.15.145:0/2170984491 wait complete. 2023-08-25T05:00:43.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.281+0000 7fb101eb6b80 1 -- start start 2023-08-25T05:00:43.295 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 Processor -- start 2023-08-25T05:00:43.302 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b38b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.303 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97b39890 con 0x560e97b2f500 2023-08-25T05:00:43.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97b39a00 con 0x560e97b2fe20 2023-08-25T05:00:43.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e97b97560 con 0x560e97b29c40 2023-08-25T05:00:43.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb101eb6b80 10 client.0 _ll_register_callbacks cb 0x560e97963a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T05:00:43.304 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35540/0 (socket says 172.21.15.145:35540) 2023-08-25T05:00:43.305 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.282+0000 7fb0e7fff700 1 -- 172.21.15.145:0/625485900 learned_addr learned my addr 172.21.15.145:0/625485900 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b38b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0ecb9c700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 -- 172.21.15.145:0/625485900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 msgr2=0x560e97b391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.306 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 -- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 msgr2=0x560e97b38b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T05:00:43.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b38b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 -- 172.21.15.145:0/625485900 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc021040 con 0x560e97b29c40 2023-08-25T05:00:43.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0e7fff700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80123f0 tx=0x7fb0d80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4d279d139482be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2023-08-25T05:00:43.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8029880 con 0x560e97b29c40 2023-08-25T05:00:43.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fb0d8026040 con 0x560e97b29c40 2023-08-25T05:00:43.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.283+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e97b1c8b0 con 0x560e97b29c40 2023-08-25T05:00:43.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d80299e0 con 0x560e97b29c40 2023-08-25T05:00:43.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 10 client.26320 fetch_fsmap learned FSMap version 35 2023-08-25T05:00:43.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560e978ce250 con 0x560e97b29c40 2023-08-25T05:00:43.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e97b1c8b0 con 0x560e97b29c40 2023-08-25T05:00:43.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb0d8005020 con 0x560e97b29c40 2023-08-25T05:00:43.311 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 35 2023-08-25T05:00:43.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2023-08-25T05:00:43.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 20 client.26320 populate_metadata read hostname 'smithi145' 2023-08-25T05:00:43.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b98d40 0x560e97b9b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 4 client.26320 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-08-25T05:00:43.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 --> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e97a74970 con 0x560e97b98d40 2023-08-25T05:00:43.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.284+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b98d40 0x560e97b9b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb0e77fe700 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b98d40 0x560e97b9b160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de29a111857ddde server_cookie=b237e8161ff2303 in_seq=0 out_seq=0 2023-08-25T05:00:43.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb0c77fe700 10 client.26320 ms_handle_connect on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:43.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb0c77fe700 1 -- 172.21.15.145:0/625485900 <== mds.0 v2:172.21.15.145:6840/3986940142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560e97a74970 con 0x560e97b98d40 2023-08-25T05:00:43.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb0c77fe700 10 client.26320 handle_command_reply: tid=0 2023-08-25T05:00:43.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 client.26320 shutdown 2023-08-25T05:00:43.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 msgr2=0x560e97b3a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80123f0 tx=0x7fb0d80047c0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 shutdown_connections 2023-08-25T05:00:43.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560e97b2fe20 0x560e97b391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:6840/3986940142,v1:172.21.15.145:6841/3986940142] conn(0x560e97b98d40 0x560e97b9b160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560e97b29c40 0x560e97b3a720 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 --2- 172.21.15.145:0/625485900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560e97b2f500 0x560e97b38b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb0c77fe700 0 client.26320 ms_handle_reset on v2:172.21.15.145:6840/3986940142 2023-08-25T05:00:43.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.285+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 >> 172.21.15.145:0/625485900 conn(0x560e97a9e0e0 msgr2=0x560e97b2cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 shutdown_connections 2023-08-25T05:00:43.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb101eb6b80 1 -- 172.21.15.145:0/625485900 wait complete. 2023-08-25T05:00:43.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb101eb6b80 20 client.26320 trim_cache size 0 max 16384 2023-08-25T05:00:43.319 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:43.319 INFO:teuthology.orchestra.run.smithi145.stdout: "status": "scrub active (1 inodes in the stack)", 2023-08-25T05:00:43.319 INFO:teuthology.orchestra.run.smithi145.stdout: "scrubs": { 2023-08-25T05:00:43.319 INFO:teuthology.orchestra.run.smithi145.stdout: "47f28dd1-b867-4692-a4cb-0363618a9e8e": { 2023-08-25T05:00:43.320 INFO:teuthology.orchestra.run.smithi145.stdout: "path": "/", 2023-08-25T05:00:43.320 INFO:teuthology.orchestra.run.smithi145.stdout: "tag": "47f28dd1-b867-4692-a4cb-0363618a9e8e", 2023-08-25T05:00:43.320 INFO:teuthology.orchestra.run.smithi145.stdout: "options": "recursive,force" 2023-08-25T05:00:43.321 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:43.321 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:43.321 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:00:43.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 msgr2=0x7fb0d00a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 0x7fb0d00a2620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005fd0 tx=0x7fb0d8013840 comp rx=0 tx=0).stop 2023-08-25T05:00:43.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 msgr2=0x7fb0e8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.286+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004a50 tx=0x7fb0e0004a80 comp rx=0 tx=0).stop 2023-08-25T05:00:43.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 shutdown_connections 2023-08-25T05:00:43.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e8108950 0x7fb0e8198720 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0e8102940 0x7fb0e819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fb0d00a0160 0x7fb0d00a2620 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 --2- 172.21.15.145:0/3639696850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0e8103300 0x7fb0e819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.326 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 >> 172.21.15.145:0/3639696850 conn(0x7fb0e80fe3b0 msgr2=0x7fb0e8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 shutdown_connections 2023-08-25T05:00:43.327 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.287+0000 7fb0ed39d700 1 -- 172.21.15.145:0/3639696850 wait complete. 2023-08-25T05:00:43.328 INFO:tasks.cephfs.filesystem:scrub status for tag:47f28dd1-b867-4692-a4cb-0363618a9e8e - {'path': '/', 'tag': '47f28dd1-b867-4692-a4cb-0363618a9e8e', 'options': 'recursive,force'} 2023-08-25T05:00:43.329 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/filesystem.py", line 1727, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-08-25T05:00:43.342 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-08-25T05:00:43.348 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-08-25T05:00:43.365 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T05:00:43.587 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/1343695345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 msgr2=0x7f9e3410f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/1343695345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3410f970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009f90 tx=0x7f9e2c00b330 comp rx=0 tx=0).stop 2023-08-25T05:00:43.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/1343695345 shutdown_connections 2023-08-25T05:00:43.588 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/1343695345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3410f970 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/1343695345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34108ab0 0x7f9e34108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/1343695345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34102aa0 0x7f9e34102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.589 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/1343695345 >> 172.21.15.145:0/1343695345 conn(0x7f9e340fe570 msgr2=0x7f9e34100990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/1343695345 shutdown_connections 2023-08-25T05:00:43.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.586+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/1343695345 wait complete. 2023-08-25T05:00:43.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.587+0000 7f9e3a5ff700 1 Processor -- start 2023-08-25T05:00:43.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.587+0000 7f9e3a5ff700 1 -- start start 2023-08-25T05:00:43.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34102aa0 0x7f9e3419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34108ab0 0x7f9e341988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34198f60 con 0x7f9e34108ab0 2023-08-25T05:00:43.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e341990d0 con 0x7f9e34102aa0 2023-08-25T05:00:43.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e3a5ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34199290 con 0x7f9e34103460 2023-08-25T05:00:43.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35554/0 (socket says 172.21.15.145:35554) 2023-08-25T05:00:43.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 -- 172.21.15.145:0/3343759606 learned_addr learned my addr 172.21.15.145:0/3343759606 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:43.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e33fff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34102aa0 0x7f9e3419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e38b9c700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34108ab0 0x7f9e341988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 -- 172.21.15.145:0/3343759606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34102aa0 msgr2=0x7f9e3419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34102aa0 0x7f9e3419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.588+0000 7f9e337fe700 1 -- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34108ab0 msgr2=0x7f9e341988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e337fe700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34108ab0 0x7f9e341988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e337fe700 1 -- 172.21.15.145:0/3343759606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009c40 con 0x7f9e34103460 2023-08-25T05:00:43.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e337fe700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e9c0 tx=0x7f9e2800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004a30 con 0x7f9e34103460 2023-08-25T05:00:43.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e2800cc80 con 0x7f9e34103460 2023-08-25T05:00:43.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e34199570 con 0x7f9e34103460 2023-08-25T05:00:43.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280054e0 con 0x7f9e34103460 2023-08-25T05:00:43.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.589+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e341a5870 con 0x7f9e34103460 2023-08-25T05:00:43.604 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.590+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e34067b60 con 0x7f9e34103460 2023-08-25T05:00:43.605 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.593+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f9e28004b90 con 0x7f9e34103460 2023-08-25T05:00:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.594+0000 7f9e317fa700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 0x7f9e1c0a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.595+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f9e28014070 con 0x7f9e34103460 2023-08-25T05:00:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.595+0000 7f9e33fff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 0x7f9e1c0a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.595+0000 7f9e33fff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 0x7f9e1c0a2640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009be0 tx=0x7f9e24009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:43.607 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.596+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9e28095180 con 0x7f9e34103460 2023-08-25T05:00:43.867 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.866+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e34199e00 con 0x7f9e34103460 2023-08-25T05:00:43.868 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.868+0000 7f9e317fa700 1 -- 172.21.15.145:0/3343759606 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+25016 (secure 0 0 0) 0x7f9e280987a0 con 0x7f9e34103460 2023-08-25T05:00:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:00:43.870 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":108,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T05:00:14.829997+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4132549840":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4072524950":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6800/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/3264758001":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/459521375":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T05:00:43.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 msgr2=0x7f9e1c0a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.871 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 0x7f9e1c0a2640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009be0 tx=0x7f9e24009260 comp rx=0 tx=0).stop 2023-08-25T05:00:43.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 msgr2=0x7f9e3419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:43.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e9c0 tx=0x7f9e2800ecd0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 shutdown_connections 2023-08-25T05:00:43.872 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e34108ab0 0x7f9e341988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9e1c0a0180 0x7f9e1c0a2640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e34102aa0 0x7f9e3419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 --2- 172.21.15.145:0/3343759606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e34103460 0x7f9e3419edc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:43.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.870+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 >> 172.21.15.145:0/3343759606 conn(0x7f9e340fe570 msgr2=0x7f9e34109990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:43.873 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.871+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 shutdown_connections 2023-08-25T05:00:43.874 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:43.871+0000 7f9e3a5ff700 1 -- 172.21.15.145:0/3343759606 wait complete. 2023-08-25T05:00:43.896 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T05:00:43.897 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T05:00:44.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 -- 172.21.15.145:0/245834636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 msgr2=0x7f9f60067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.111 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 --2- 172.21.15.145:0/245834636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f60067f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009990 tx=0x7f9f50009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 -- 172.21.15.145:0/245834636 shutdown_connections 2023-08-25T05:00:44.112 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 --2- 172.21.15.145:0/245834636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 0x7f9f6010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 --2- 172.21.15.145:0/245834636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 0x7f9f60106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 --2- 172.21.15.145:0/245834636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f60067f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.109+0000 7f9f6639d700 1 -- 172.21.15.145:0/245834636 >> 172.21.15.145:0/245834636 conn(0x7f9f600750e0 msgr2=0x7f9f600754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:44.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.110+0000 7f9f6639d700 1 -- 172.21.15.145:0/245834636 shutdown_connections 2023-08-25T05:00:44.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.110+0000 7f9f6639d700 1 -- 172.21.15.145:0/245834636 wait complete. 2023-08-25T05:00:44.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 Processor -- start 2023-08-25T05:00:44.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 -- start start 2023-08-25T05:00:44.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 0x7f9f6019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 0x7f9f60198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60198df0 con 0x7f9f60068440 2023-08-25T05:00:44.118 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60198f60 con 0x7f9f60106780 2023-08-25T05:00:44.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.111+0000 7f9f6639d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60199120 con 0x7f9f60067a80 2023-08-25T05:00:44.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35576/0 (socket says 172.21.15.145:35576) 2023-08-25T05:00:44.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 -- 172.21.15.145:0/4200418137 learned_addr learned my addr 172.21.15.145:0/4200418137 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:44.119 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f64b9a700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 0x7f9f6019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f65b9c700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 0x7f9f60198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 -- 172.21.15.145:0/4200418137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 msgr2=0x7f9f60198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 0x7f9f60198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 -- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 msgr2=0x7f9f6019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 0x7f9f6019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 -- 172.21.15.145:0/4200418137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500095f0 con 0x7f9f60067a80 2023-08-25T05:00:44.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f6539b700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d980 tx=0x7f9f5c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:44.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.112+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00bb60 con 0x7f9f60067a80 2023-08-25T05:00:44.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.113+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5c007640 con 0x7f9f60067a80 2023-08-25T05:00:44.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.113+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c004e90 con 0x7f9f60067a80 2023-08-25T05:00:44.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.113+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f60199400 con 0x7f9f60067a80 2023-08-25T05:00:44.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.113+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601a5700 con 0x7f9f60067a80 2023-08-25T05:00:44.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.114+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f60067630 con 0x7f9f60067a80 2023-08-25T05:00:44.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.116+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f9f5c00bcc0 con 0x7f9f60067a80 2023-08-25T05:00:44.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.117+0000 7f9f567fc700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 0x7f9f4c0a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.117+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f9f5c0cb780 con 0x7f9f60067a80 2023-08-25T05:00:44.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.117+0000 7f9f64b9a700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 0x7f9f4c0a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.118+0000 7f9f64b9a700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 0x7f9f4c0a26a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b2e0 tx=0x7f9f50005fb0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:44.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.121+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f9f5c094e30 con 0x7f9f60067a80 2023-08-25T05:00:44.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:43 smithi162 ceph-mon[130941]: pgmap v490: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:44.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:43 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3343759606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:00:44.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:43 smithi183 ceph-mon[131414]: pgmap v490: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:44.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:43 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3343759606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:00:44.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:43 smithi145 ceph-mon[97681]: pgmap v490: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:44.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:43 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3343759606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:00:44.425 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.425+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f601108b0 con 0x7f9f60067a80 2023-08-25T05:00:44.426 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.425+0000 7f9f567fc700 1 -- 172.21.15.145:0/4200418137 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f5c00de60 con 0x7f9f60067a80 2023-08-25T05:00:44.426 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T05:00:44.429 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 msgr2=0x7f9f4c0a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 0x7f9f4c0a26a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b2e0 tx=0x7f9f50005fb0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 msgr2=0x7f9f6019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d980 tx=0x7f9f5c00dd40 comp rx=0 tx=0).stop 2023-08-25T05:00:44.430 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 shutdown_connections 2023-08-25T05:00:44.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f9f4c0a01e0 0x7f9f4c0a26a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f60106780 0x7f9f60198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.431 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60067a80 0x7f9f6019e710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 --2- 172.21.15.145:0/4200418137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f60068440 0x7f9f6019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.428+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 >> 172.21.15.145:0/4200418137 conn(0x7f9f600750e0 msgr2=0x7f9f6010b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:44.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.429+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 shutdown_connections 2023-08-25T05:00:44.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.429+0000 7f9f6639d700 1 -- 172.21.15.145:0/4200418137 wait complete. 2023-08-25T05:00:44.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T05:00:44.691 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 -- 172.21.15.145:0/3275155024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 msgr2=0x7f4954073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 --2- 172.21.15.145:0/3275155024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f4954073130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4944009990 tx=0x7f4944009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.692 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 -- 172.21.15.145:0/3275155024 shutdown_connections 2023-08-25T05:00:44.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 --2- 172.21.15.145:0/3275155024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f495410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 --2- 172.21.15.145:0/3275155024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f4954073130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 --2- 172.21.15.145:0/3275155024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 0x7f495410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 -- 172.21.15.145:0/3275155024 >> 172.21.15.145:0/3275155024 conn(0x7f49540fc3a0 msgr2=0x7f49540fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:44.693 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.690+0000 7f4959dff700 1 -- 172.21.15.145:0/3275155024 shutdown_connections 2023-08-25T05:00:44.694 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.691+0000 7f4959dff700 1 -- 172.21.15.145:0/3275155024 wait complete. 2023-08-25T05:00:44.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.692+0000 7f4959dff700 1 Processor -- start 2023-08-25T05:00:44.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.692+0000 7f4959dff700 1 -- start start 2023-08-25T05:00:44.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.696 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 0x7f495419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f4954198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954199090 con 0x7f4954073670 2023-08-25T05:00:44.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954199200 con 0x7f495410d0b0 2023-08-25T05:00:44.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4959dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541993c0 con 0x7f4954072d50 2023-08-25T05:00:44.697 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35598/0 (socket says 172.21.15.145:35598) 2023-08-25T05:00:44.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 -- 172.21.15.145:0/2417595209 learned_addr learned my addr 172.21.15.145:0/2417595209 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:44.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4952ffd700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 0x7f495419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.698 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4953fff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f4954198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 -- 172.21.15.145:0/2417595209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 msgr2=0x7f4954198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f4954198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 -- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 msgr2=0x7f495419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.699 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 0x7f495419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f49537fe700 1 -- 172.21.15.145:0/2417595209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440095f0 con 0x7f4954072d50 2023-08-25T05:00:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.693+0000 7f4953fff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f4954198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:00:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f49537fe700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c00 tx=0x7f49440047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494401d070 con 0x7f4954072d50 2023-08-25T05:00:44.700 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f494400fb80 con 0x7f4954072d50 2023-08-25T05:00:44.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4954199640 con 0x7f4954072d50 2023-08-25T05:00:44.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440173f0 con 0x7f4954072d50 2023-08-25T05:00:44.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.694+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49541a59a0 con 0x7f4954072d50 2023-08-25T05:00:44.701 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.695+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49541082c0 con 0x7f4954072d50 2023-08-25T05:00:44.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.698+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f494400fcf0 con 0x7f4954072d50 2023-08-25T05:00:44.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.699+0000 7f4950ff9700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 0x7f493c0a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:44.702 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.699+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f49440ccd30 con 0x7f4954072d50 2023-08-25T05:00:44.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.699+0000 7f4952ffd700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 0x7f493c0a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:44.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.700+0000 7f4952ffd700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 0x7f493c0a2580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f495410e640 tx=0x7f4948009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:44.703 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.703+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4944096470 con 0x7f4954072d50 2023-08-25T05:00:44.960 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.959+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49541129d0 con 0x7f4954072d50 2023-08-25T05:00:44.961 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.959+0000 7f4950ff9700 1 -- 172.21.15.145:0/2417595209 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4944099a90 con 0x7f4954072d50 2023-08-25T05:00:44.961 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T05:00:44.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 msgr2=0x7f493c0a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.963 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 0x7f493c0a2580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f495410e640 tx=0x7f4948009260 comp rx=0 tx=0).stop 2023-08-25T05:00:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 msgr2=0x7f495419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c00 tx=0x7f49440047b0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 shutdown_connections 2023-08-25T05:00:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f493c0a00c0 0x7f493c0a2580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.964 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f495410d0b0 0x7f4954198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4954072d50 0x7f495419e9b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 --2- 172.21.15.145:0/2417595209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4954073670 0x7f495419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:44.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 >> 172.21.15.145:0/2417595209 conn(0x7f49540fc3a0 msgr2=0x7f495410f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:44.965 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 shutdown_connections 2023-08-25T05:00:44.966 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:44.962+0000 7f4959dff700 1 -- 172.21.15.145:0/2417595209 wait complete. 2023-08-25T05:00:44.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T05:00:45.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:44 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4200418137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:00:45.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:44 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4200418137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:00:45.250 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 -- 172.21.15.145:0/39264651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3581006b0 msgr2=0x7fd358100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.251 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 --2- 172.21.15.145:0/39264651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3581006b0 0x7fd358100b30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009990 tx=0x7fd34c009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 -- 172.21.15.145:0/39264651 shutdown_connections 2023-08-25T05:00:45.252 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 --2- 172.21.15.145:0/39264651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd358101070 0x7fd35810d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.253 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 --2- 172.21.15.145:0/39264651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581066c0 0x7fd358106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.253 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 --2- 172.21.15.145:0/39264651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3581006b0 0x7fd358100b30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.253 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 -- 172.21.15.145:0/39264651 >> 172.21.15.145:0/39264651 conn(0x7fd3580fc160 msgr2=0x7fd3580fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:45.253 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 -- 172.21.15.145:0/39264651 shutdown_connections 2023-08-25T05:00:45.254 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.246+0000 7fd35d39d700 1 -- 172.21.15.145:0/39264651 wait complete. 2023-08-25T05:00:45.254 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.247+0000 7fd35d39d700 1 Processor -- start 2023-08-25T05:00:45.254 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.247+0000 7fd35d39d700 1 -- start start 2023-08-25T05:00:45.254 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 0x7fd35819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.254 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3581066c0 0x7fd358198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358198d90 con 0x7fd3581066c0 2023-08-25T05:00:45.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358198f00 con 0x7fd3581006b0 2023-08-25T05:00:45.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35d39d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581990c0 con 0x7fd358101070 2023-08-25T05:00:45.255 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd3577fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd357fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 0x7fd35819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd3577fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35608/0 (socket says 172.21.15.145:35608) 2023-08-25T05:00:45.256 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd35cb9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3581066c0 0x7fd358198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd3577fe700 1 -- 172.21.15.145:0/1104075498 learned_addr learned my addr 172.21.15.145:0/1104075498 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:45.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.248+0000 7fd3577fe700 1 -- 172.21.15.145:0/1104075498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 msgr2=0x7fd35819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3577fe700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 0x7fd35819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3577fe700 1 -- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3581066c0 msgr2=0x7fd358198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.257 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3577fe700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3581066c0 0x7fd358198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3577fe700 1 -- 172.21.15.145:0/1104075498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0095f0 con 0x7fd358101070 2023-08-25T05:00:45.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd357fff700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 0x7fd35819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:00:45.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3577fe700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005910 tx=0x7fd34c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:45.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c01d070 con 0x7fd358101070 2023-08-25T05:00:45.258 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34c022470 con 0x7fd358101070 2023-08-25T05:00:45.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd358199340 con 0x7fd358101070 2023-08-25T05:00:45.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c017650 con 0x7fd358101070 2023-08-25T05:00:45.259 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.249+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3581a56a0 con 0x7fd358101070 2023-08-25T05:00:45.260 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.251+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd358067a80 con 0x7fd358101070 2023-08-25T05:00:45.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.252+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fd34c0225e0 con 0x7fd358101070 2023-08-25T05:00:45.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.254+0000 7fd3557fa700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 0x7fd3400a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.254+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fd34c0ccc50 con 0x7fd358101070 2023-08-25T05:00:45.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.254+0000 7fd357fff700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 0x7fd3400a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.255+0000 7fd357fff700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 0x7fd3400a26a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd348009fd0 tx=0x7fd348009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:45.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.257+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fd34c096390 con 0x7fd358101070 2023-08-25T05:00:45.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:44 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4200418137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:00:45.520 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.519+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd358110680 con 0x7fd358101070 2023-08-25T05:00:45.521 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.520+0000 7fd3557fa700 1 -- 172.21.15.145:0/1104075498 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd34c027020 con 0x7fd358101070 2023-08-25T05:00:45.522 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T05:00:45.523 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.522+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 msgr2=0x7fd3400a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 0x7fd3400a26a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd348009fd0 tx=0x7fd348009260 comp rx=0 tx=0).stop 2023-08-25T05:00:45.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 msgr2=0x7fd35819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005910 tx=0x7fd34c005720 comp rx=0 tx=0).stop 2023-08-25T05:00:45.524 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 shutdown_connections 2023-08-25T05:00:45.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3581066c0 0x7fd358198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fd3400a01e0 0x7fd3400a26a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3581006b0 0x7fd35819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.525 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 --2- 172.21.15.145:0/1104075498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd358101070 0x7fd35819ebf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 >> 172.21.15.145:0/1104075498 conn(0x7fd3580fc160 msgr2=0x7fd3580fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:45.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 shutdown_connections 2023-08-25T05:00:45.526 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.523+0000 7fd35d39d700 1 -- 172.21.15.145:0/1104075498 wait complete. 2023-08-25T05:00:45.545 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T05:00:45.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 -- 172.21.15.145:0/697084795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c101a40 msgr2=0x7f434c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.799 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 --2- 172.21.15.145:0/697084795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c101a40 0x7f434c101e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f433c009990 tx=0x7f433c009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 -- 172.21.15.145:0/697084795 shutdown_connections 2023-08-25T05:00:45.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 --2- 172.21.15.145:0/697084795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c10ae60 0x7f434c10d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 --2- 172.21.15.145:0/697084795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c101a40 0x7f434c101e20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.800 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 --2- 172.21.15.145:0/697084795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c102360 0x7f434c10a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 -- 172.21.15.145:0/697084795 >> 172.21.15.145:0/697084795 conn(0x7f434c0fb260 msgr2=0x7f434c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:45.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 -- 172.21.15.145:0/697084795 shutdown_connections 2023-08-25T05:00:45.801 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.798+0000 7f4351dff700 1 -- 172.21.15.145:0/697084795 wait complete. 2023-08-25T05:00:45.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.799+0000 7f4351dff700 1 Processor -- start 2023-08-25T05:00:45.803 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.799+0000 7f4351dff700 1 -- start start 2023-08-25T05:00:45.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c101a40 0x7f434c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 0x7f434c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.804 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c199150 con 0x7f434c10ae60 2023-08-25T05:00:45.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c1992c0 con 0x7f434c101a40 2023-08-25T05:00:45.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f4351dff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c199480 con 0x7f434c102360 2023-08-25T05:00:45.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35636/0 (socket says 172.21.15.145:35636) 2023-08-25T05:00:45.805 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 -- 172.21.15.145:0/693177211 learned_addr learned my addr 172.21.15.145:0/693177211 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:45.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434b7fe700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c101a40 0x7f434c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434bfff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 0x7f434c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 -- 172.21.15.145:0/693177211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c101a40 msgr2=0x7f434c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.806 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c101a40 0x7f434c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 -- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 msgr2=0x7f434c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:45.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 0x7f434c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:45.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434affd700 1 -- 172.21.15.145:0/693177211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0095f0 con 0x7f434c102360 2023-08-25T05:00:45.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.800+0000 7f434bfff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 0x7f434c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:00:45.807 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f434affd700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:45.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000c9c0 con 0x7f434c102360 2023-08-25T05:00:45.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4340010430 con 0x7f434c102360 2023-08-25T05:00:45.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340018680 con 0x7f434c102360 2023-08-25T05:00:45.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c199760 con 0x7f434c102360 2023-08-25T05:00:45.808 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.801+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c1a5a60 con 0x7f434c102360 2023-08-25T05:00:45.809 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.802+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c108390 con 0x7f434c102360 2023-08-25T05:00:45.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.804+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f4340010860 con 0x7f434c102360 2023-08-25T05:00:45.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.806+0000 7f4348ff9700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 0x7f43340a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:45.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.806+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f4340014070 con 0x7f434c102360 2023-08-25T05:00:45.810 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.806+0000 7f434b7fe700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 0x7f43340a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:45.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.807+0000 7f434b7fe700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 0x7f43340a2600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b2e0 tx=0x7f433c005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:45.811 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:45.809+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f4340051e00 con 0x7f434c102360 2023-08-25T05:00:46.045 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.044+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f434c110810 con 0x7f434c102360 2023-08-25T05:00:46.046 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.044+0000 7f4348ff9700 1 -- 172.21.15.145:0/693177211 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4340095430 con 0x7f434c102360 2023-08-25T05:00:46.046 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T05:00:46.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 msgr2=0x7f43340a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:46.048 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 0x7f43340a2600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b2e0 tx=0x7f433c005fd0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.049 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 msgr2=0x7f434c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:46.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000ed00 comp rx=0 tx=0).stop 2023-08-25T05:00:46.053 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 shutdown_connections 2023-08-25T05:00:46.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f434c10ae60 0x7f434c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f43340a0140 0x7f43340a2600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434c101a40 0x7f434c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 --2- 172.21.15.145:0/693177211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434c102360 0x7f434c19efb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.054 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 >> 172.21.15.145:0/693177211 conn(0x7f434c0fb260 msgr2=0x7f434c10d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:46.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 shutdown_connections 2023-08-25T05:00:46.055 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.047+0000 7f4351dff700 1 -- 172.21.15.145:0/693177211 wait complete. 2023-08-25T05:00:46.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-08-25T05:00:46.144 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:45 smithi145 ceph-mon[97681]: pgmap v491: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:46.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:45 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2417595209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.145 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:45 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1104075498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:45 smithi162 ceph-mon[130941]: pgmap v491: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:46.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:45 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2417595209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:45 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1104075498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:45 smithi183 ceph-mon[131414]: pgmap v491: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:46.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:45 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2417595209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:45 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1104075498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:00:46.307 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 -- 172.21.15.145:0/423638863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968100890 msgr2=0x7f1968100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:46.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 --2- 172.21.15.145:0/423638863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968100890 0x7f1968100d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f195c009990 tx=0x7f195c009ca0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.308 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 -- 172.21.15.145:0/423638863 shutdown_connections 2023-08-25T05:00:46.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 --2- 172.21.15.145:0/423638863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1968101250 0x7f196810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 --2- 172.21.15.145:0/423638863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19681068a0 0x7f1968106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 --2- 172.21.15.145:0/423638863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968100890 0x7f1968100d10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 -- 172.21.15.145:0/423638863 >> 172.21.15.145:0/423638863 conn(0x7f19680fc340 msgr2=0x7f19680fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:46.309 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 -- 172.21.15.145:0/423638863 shutdown_connections 2023-08-25T05:00:46.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.303+0000 7f196cbff700 1 -- 172.21.15.145:0/423638863 wait complete. 2023-08-25T05:00:46.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.304+0000 7f196cbff700 1 Processor -- start 2023-08-25T05:00:46.310 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 -- start start 2023-08-25T05:00:46.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:46.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:46.312 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 0x7f1968196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1968196e00 con 0x7f1968100890 2023-08-25T05:00:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1968196f70 con 0x7f19681068a0 2023-08-25T05:00:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196cbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1968197130 con 0x7f1968101250 2023-08-25T05:00:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:35658/0 (socket says 172.21.15.145:35658) 2023-08-25T05:00:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f1965d9b700 1 -- 172.21.15.145:0/2132404083 learned_addr learned my addr 172.21.15.145:0/2132404083 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:00:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.305+0000 7f196659c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58264/0 (socket says 172.21.15.145:58264) 2023-08-25T05:00:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1966d9d700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 0x7f1968196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 -- 172.21.15.145:0/2132404083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 msgr2=0x7f1968196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 0x7f1968196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 -- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 msgr2=0x7f196819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 -- 172.21.15.145:0/2132404083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0095f0 con 0x7f1968101250 2023-08-25T05:00:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1966d9d700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 0x7f1968196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T05:00:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f196659c700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:00:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f1965d9b700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f195c005910 tx=0x7f195c004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c01d070 con 0x7f1968101250 2023-08-25T05:00:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f195c022470 con 0x7f1968101250 2023-08-25T05:00:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c017650 con 0x7f1968101250 2023-08-25T05:00:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.306+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19681973b0 con 0x7f1968101250 2023-08-25T05:00:46.321 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.307+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681a3710 con 0x7f1968101250 2023-08-25T05:00:46.322 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.308+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968067710 con 0x7f1968101250 2023-08-25T05:00:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.309+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f195c0225e0 con 0x7f1968101250 2023-08-25T05:00:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.311+0000 7f19577fe700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 0x7f19500a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:00:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.311+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f195c0ccb70 con 0x7f1968101250 2023-08-25T05:00:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.311+0000 7f196659c700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 0x7f19500a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:00:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.312+0000 7f196659c700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 0x7f19500a2640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:00:46.325 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.315+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f195c05ce00 con 0x7f1968101250 2023-08-25T05:00:46.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.621+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f19681108d0 con 0x7f1968101250 2023-08-25T05:00:46.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.630+0000 7f19577fe700 1 -- 172.21.15.145:0/2132404083 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+728397 (secure 0 0 0) 0x7f195c096310 con 0x7f1968101250 2023-08-25T05:00:46.641 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:00:46.642 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:46.642 INFO:teuthology.orchestra.run.smithi145.stdout: "epoch": 24, 2023-08-25T05:00:46.642 INFO:teuthology.orchestra.run.smithi145.stdout: "active_gid": 15495, 2023-08-25T05:00:46.642 INFO:teuthology.orchestra.run.smithi145.stdout: "active_name": "x", 2023-08-25T05:00:46.643 INFO:teuthology.orchestra.run.smithi145.stdout: "active_addrs": { 2023-08-25T05:00:46.643 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:46.644 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.644 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:46.645 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:6800", 2023-08-25T05:00:46.646 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 2587440661 2023-08-25T05:00:46.646 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.646 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.647 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2023-08-25T05:00:46.647 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:6801", 2023-08-25T05:00:46.647 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 2587440661 2023-08-25T05:00:46.648 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:46.648 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:46.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.649 INFO:teuthology.orchestra.run.smithi145.stdout: "active_addr": "172.21.15.145:6801/2587440661", 2023-08-25T05:00:46.649 INFO:teuthology.orchestra.run.smithi145.stdout: "active_change": "2023-08-25T04:45:04.447889+0000", 2023-08-25T05:00:46.649 INFO:teuthology.orchestra.run.smithi145.stdout: "active_mgr_features": 4540138322906710015, 2023-08-25T05:00:46.649 INFO:teuthology.orchestra.run.smithi145.stdout: "available": true, 2023-08-25T05:00:46.655 INFO:teuthology.orchestra.run.smithi145.stdout: "standbys": [ 2023-08-25T05:00:46.655 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.655 INFO:teuthology.orchestra.run.smithi145.stdout: "gid": 25490, 2023-08-25T05:00:46.656 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "y", 2023-08-25T05:00:46.656 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_features": 4540138322906710015, 2023-08-25T05:00:46.656 INFO:teuthology.orchestra.run.smithi145.stdout: "available_modules": [ 2023-08-25T05:00:46.657 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.657 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "alerts", 2023-08-25T05:00:46.657 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:46.659 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:46.660 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.660 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.661 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.661 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:46.662 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.662 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.662 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.663 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T05:00:46.663 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.663 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.664 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.664 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:46.664 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:46.665 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.665 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.665 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.665 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.665 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.666 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.666 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.666 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.666 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.666 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.667 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.667 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.667 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.667 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.667 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.668 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.668 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.668 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.669 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:46.669 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:46.669 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.669 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.670 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.670 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.670 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.670 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.670 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.671 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.671 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.671 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.671 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.672 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:46.673 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.673 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.674 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.674 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.675 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.675 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.677 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.678 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.678 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.678 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.679 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.680 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.680 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.681 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.681 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:46.682 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:46.682 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.683 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.683 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.684 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.685 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.686 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.687 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.687 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.688 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.689 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.689 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.689 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_destination": { 2023-08-25T05:00:46.690 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_destination", 2023-08-25T05:00:46.690 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.690 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.690 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.690 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.691 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.691 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.691 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.691 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email address to send alerts to", 2023-08-25T05:00:46.691 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.692 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.692 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.692 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_from_name": { 2023-08-25T05:00:46.692 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_from_name", 2023-08-25T05:00:46.693 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.693 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.693 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.693 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "Ceph", 2023-08-25T05:00:46.694 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.694 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.694 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.694 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email From: name", 2023-08-25T05:00:46.695 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.696 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.696 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.697 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_host": { 2023-08-25T05:00:46.698 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_host", 2023-08-25T05:00:46.698 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.699 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.699 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.699 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.700 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.700 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.700 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.700 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP server", 2023-08-25T05:00:46.700 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.701 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.701 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.701 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_password": { 2023-08-25T05:00:46.702 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_password", 2023-08-25T05:00:46.702 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.702 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.702 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.702 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.703 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.703 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.703 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.703 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Password to authenticate with", 2023-08-25T05:00:46.703 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.704 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.704 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.704 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_port": { 2023-08-25T05:00:46.705 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_port", 2023-08-25T05:00:46.705 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.705 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.705 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.705 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "465", 2023-08-25T05:00:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP port", 2023-08-25T05:00:46.706 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_sender": { 2023-08-25T05:00:46.707 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_sender", 2023-08-25T05:00:46.708 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.708 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.708 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.711 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.712 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.713 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.713 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.714 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP envelope sender", 2023-08-25T05:00:46.714 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.715 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.715 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.716 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_ssl": { 2023-08-25T05:00:46.718 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_ssl", 2023-08-25T05:00:46.718 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.718 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.720 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.721 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.721 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.721 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.721 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.722 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T05:00:46.722 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.722 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.722 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.723 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_user": { 2023-08-25T05:00:46.723 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_user", 2023-08-25T05:00:46.724 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.724 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "User to authenticate as", 2023-08-25T05:00:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "balancer", 2023-08-25T05:00:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "active": { 2023-08-25T05:00:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "active", 2023-08-25T05:00:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T05:00:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_time": { 2023-08-25T05:00:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_time", 2023-08-25T05:00:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0000", 2023-08-25T05:00:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T05:00:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.735 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_weekday": { 2023-08-25T05:00:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_weekday", 2023-08-25T05:00:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:46.736 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:46.737 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T05:00:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_max_iterations": { 2023-08-25T05:00:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_max_iterations", 2023-08-25T05:00:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.739 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "25", 2023-08-25T05:00:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "250", 2023-08-25T05:00:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.740 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_metrics": { 2023-08-25T05:00:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_metrics", 2023-08-25T05:00:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T05:00:46.742 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.743 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.743 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.743 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T05:00:46.743 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T05:00:46.743 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.744 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.744 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.744 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_step": { 2023-08-25T05:00:46.744 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_step", 2023-08-25T05:00:46.744 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:46.745 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.746 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.746 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.5", 2023-08-25T05:00:46.746 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0.001", 2023-08-25T05:00:46.746 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "0.999", 2023-08-25T05:00:46.746 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.747 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "aggressiveness of optimization", 2023-08-25T05:00:46.747 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T05:00:46.747 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.747 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.748 INFO:teuthology.orchestra.run.smithi145.stdout: "end_time": { 2023-08-25T05:00:46.748 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_time", 2023-08-25T05:00:46.748 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.748 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.748 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.749 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2359", 2023-08-25T05:00:46.749 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.749 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.749 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.749 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T05:00:46.750 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:46.750 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.750 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.750 INFO:teuthology.orchestra.run.smithi145.stdout: "end_weekday": { 2023-08-25T05:00:46.751 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_weekday", 2023-08-25T05:00:46.751 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:46.751 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.751 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.752 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:46.752 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:46.752 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:46.752 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.753 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T05:00:46.753 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:46.753 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.753 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.754 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:46.754 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:46.754 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.754 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.754 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.755 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.755 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.755 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.756 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.756 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.756 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.756 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.756 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.757 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.757 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.757 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.757 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.757 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.758 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.758 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.758 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:46.758 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.760 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.760 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.760 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.760 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.761 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.761 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.761 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:46.761 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:46.762 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.762 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.762 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.762 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:46.762 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.763 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.764 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.764 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.764 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.764 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.764 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.765 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.765 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.766 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:46.766 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:46.766 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.766 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.766 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.767 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.767 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.767 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.767 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.767 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.768 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.768 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.768 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.768 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.768 INFO:teuthology.orchestra.run.smithi145.stdout: "min_score": { 2023-08-25T05:00:46.769 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_score", 2023-08-25T05:00:46.769 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:46.769 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.769 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.769 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.770 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.773 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.774 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.774 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:46.774 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:46.774 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.775 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.775 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.775 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "upmap", 2023-08-25T05:00:46.775 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.775 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.776 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.776 INFO:teuthology.orchestra.run.smithi145.stdout: "crush-compat", 2023-08-25T05:00:46.776 INFO:teuthology.orchestra.run.smithi145.stdout: "none", 2023-08-25T05:00:46.776 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap" 2023-08-25T05:00:46.776 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.777 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Balancer mode", 2023-08-25T05:00:46.777 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.777 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.777 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.777 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.778 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ids": { 2023-08-25T05:00:46.778 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_ids", 2023-08-25T05:00:46.778 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.778 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.778 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.779 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.779 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.779 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.779 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.780 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T05:00:46.780 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.780 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.780 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.781 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:46.781 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:46.781 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.781 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.782 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.782 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:46.782 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.782 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.782 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.783 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T05:00:46.783 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.783 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.784 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.784 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_deviation": { 2023-08-25T05:00:46.784 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_deviation", 2023-08-25T05:00:46.785 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.785 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.785 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.785 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:46.785 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:46.786 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.786 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.786 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T05:00:46.786 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T05:00:46.786 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.787 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.787 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_optimizations": { 2023-08-25T05:00:46.787 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_optimizations", 2023-08-25T05:00:46.787 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:46.788 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.788 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.788 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:46.788 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.788 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.789 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.789 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T05:00:46.789 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.789 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:46.790 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cephadm", 2023-08-25T05:00:46.791 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:46.791 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:46.791 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:46.791 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_down_multiplier": { 2023-08-25T05:00:46.792 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_down_multiplier", 2023-08-25T05:00:46.792 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:46.792 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.792 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.792 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:46.793 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.793 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.793 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.793 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T05:00:46.793 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.794 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.794 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.794 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_refresh_rate": { 2023-08-25T05:00:46.794 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_refresh_rate", 2023-08-25T05:00:46.795 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.795 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.795 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.795 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "20", 2023-08-25T05:00:46.795 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.796 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.800 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.801 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T05:00:46.801 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.801 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.801 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.802 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_starting_port": { 2023-08-25T05:00:46.802 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_starting_port", 2023-08-25T05:00:46.802 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.802 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.803 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.803 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4721", 2023-08-25T05:00:46.803 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.803 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.804 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.804 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T05:00:46.805 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.805 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.805 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.806 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_ptrace": { 2023-08-25T05:00:46.807 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_ptrace", 2023-08-25T05:00:46.807 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.808 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.808 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.808 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.809 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.809 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.809 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.809 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T05:00:46.810 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T05:00:46.810 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.811 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.811 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.811 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_interval": { 2023-08-25T05:00:46.812 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_interval", 2023-08-25T05:00:46.812 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.813 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.813 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.813 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:46.814 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.814 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.814 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.815 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T05:00:46.815 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.815 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.815 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.816 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.816 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_memory_target_ratio": { 2023-08-25T05:00:46.817 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T05:00:46.817 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:46.818 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.818 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.818 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.7", 2023-08-25T05:00:46.819 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.819 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.819 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.820 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T05:00:46.820 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.821 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.821 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.821 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.822 INFO:teuthology.orchestra.run.smithi145.stdout: "cgroups_split": { 2023-08-25T05:00:46.822 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cgroups_split", 2023-08-25T05:00:46.823 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.823 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.824 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.825 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.825 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.826 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.826 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.827 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T05:00:46.828 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.828 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.829 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.830 INFO:teuthology.orchestra.run.smithi145.stdout: "config_checks_enabled": { 2023-08-25T05:00:46.830 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_checks_enabled", 2023-08-25T05:00:46.831 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.831 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.832 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.832 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.833 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.833 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.834 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.834 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T05:00:46.834 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.835 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.835 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.835 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.835 INFO:teuthology.orchestra.run.smithi145.stdout: "config_dashboard": { 2023-08-25T05:00:46.836 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_dashboard", 2023-08-25T05:00:46.836 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.836 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.836 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.837 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.837 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.837 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.838 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.838 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T05:00:46.838 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.839 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.839 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_alertmanager": { 2023-08-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_alertmanager", 2023-08-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.842 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.842 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.842 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:46.843 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.843 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.843 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.844 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.844 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_base": { 2023-08-25T05:00:46.844 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_base", 2023-08-25T05:00:46.845 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.845 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.845 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.845 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T05:00:46.846 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.846 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.846 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.846 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Container image name, without the tag", 2023-08-25T05:00:46.846 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.847 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.847 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.847 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.847 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_elasticsearch": { 2023-08-25T05:00:46.847 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_elasticsearch", 2023-08-25T05:00:46.848 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.848 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.848 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.848 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T05:00:46.848 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.849 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.849 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.849 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "elasticsearch container image", 2023-08-25T05:00:46.849 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.850 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.850 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.850 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.851 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_grafana": { 2023-08-25T05:00:46.851 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_grafana", 2023-08-25T05:00:46.851 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.851 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.852 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.852 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T05:00:46.852 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.852 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.853 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.853 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:46.853 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.853 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.853 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.854 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.854 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_haproxy": { 2023-08-25T05:00:46.855 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_haproxy", 2023-08-25T05:00:46.855 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.855 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.855 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.856 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T05:00:46.856 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.856 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.857 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.857 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "HAproxy container image", 2023-08-25T05:00:46.857 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.858 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.858 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.858 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.858 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_agent": { 2023-08-25T05:00:46.859 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_agent", 2023-08-25T05:00:46.859 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.859 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.860 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.860 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T05:00:46.860 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.861 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.861 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.862 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger agent container image", 2023-08-25T05:00:46.862 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.862 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.863 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.863 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.863 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_collector": { 2023-08-25T05:00:46.863 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_collector", 2023-08-25T05:00:46.864 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.864 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.864 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.864 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T05:00:46.865 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.865 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.866 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.866 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger collector container image", 2023-08-25T05:00:46.866 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.867 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.867 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.867 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.868 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_query": { 2023-08-25T05:00:46.868 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_query", 2023-08-25T05:00:46.868 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.869 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.869 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.870 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T05:00:46.870 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.870 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.871 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.871 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger query container image", 2023-08-25T05:00:46.871 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.871 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.872 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.872 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.872 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_keepalived": { 2023-08-25T05:00:46.873 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_keepalived", 2023-08-25T05:00:46.873 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.873 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.874 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.874 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T05:00:46.875 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.875 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.875 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.876 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Keepalived container image", 2023-08-25T05:00:46.876 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.877 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.877 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.877 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.877 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_loki": { 2023-08-25T05:00:46.877 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_loki", 2023-08-25T05:00:46.878 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.878 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.878 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.878 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T05:00:46.879 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.879 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.879 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.879 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Loki container image", 2023-08-25T05:00:46.879 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.880 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.880 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.880 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.880 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_node_exporter": { 2023-08-25T05:00:46.880 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_node_exporter", 2023-08-25T05:00:46.881 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.881 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.881 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.881 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T05:00:46.882 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.882 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.882 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.882 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:46.882 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.883 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.883 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.883 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.883 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_nvmeof": { 2023-08-25T05:00:46.883 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_nvmeof", 2023-08-25T05:00:46.884 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.884 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.885 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.885 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T05:00:46.885 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.885 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.886 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.886 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Nvme-of container image", 2023-08-25T05:00:46.886 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.886 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.886 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.887 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_prometheus": { 2023-08-25T05:00:46.887 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_prometheus", 2023-08-25T05:00:46.887 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.888 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.888 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.888 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T05:00:46.888 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.889 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.889 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.889 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:46.889 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.889 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.890 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.890 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.890 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_promtail": { 2023-08-25T05:00:46.890 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_promtail", 2023-08-25T05:00:46.890 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.891 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.891 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.891 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T05:00:46.891 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.891 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.892 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.892 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Promtail container image", 2023-08-25T05:00:46.892 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.892 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.892 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.893 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_snmp_gateway": { 2023-08-25T05:00:46.893 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_snmp_gateway", 2023-08-25T05:00:46.893 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.894 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.894 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.894 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T05:00:46.894 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.895 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.895 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.895 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SNMP Gateway container image", 2023-08-25T05:00:46.895 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.895 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.896 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.896 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.896 INFO:teuthology.orchestra.run.smithi145.stdout: "container_init": { 2023-08-25T05:00:46.896 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_init", 2023-08-25T05:00:46.897 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.897 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.897 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.897 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.897 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.898 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.898 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.899 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T05:00:46.899 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.899 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.900 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.900 INFO:teuthology.orchestra.run.smithi145.stdout: "daemon_cache_timeout": { 2023-08-25T05:00:46.900 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "daemon_cache_timeout", 2023-08-25T05:00:46.901 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.901 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.902 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.902 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:46.902 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.903 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.903 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.903 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T05:00:46.904 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.904 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi145.stdout: "default_cephadm_command_timeout": { 2023-08-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T05:00:46.906 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.906 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.906 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.906 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "900", 2023-08-25T05:00:46.906 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.907 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.907 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.907 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T05:00:46.907 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.908 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.908 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.909 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.909 INFO:teuthology.orchestra.run.smithi145.stdout: "default_registry": { 2023-08-25T05:00:46.909 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_registry", 2023-08-25T05:00:46.909 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.909 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.910 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.910 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io", 2023-08-25T05:00:46.910 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.910 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.910 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.911 INFO:teuthology.orchestra.run.smithi145.stdout: "device_cache_timeout": { 2023-08-25T05:00:46.912 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_cache_timeout", 2023-08-25T05:00:46.912 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.912 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.912 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.913 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1800", 2023-08-25T05:00:46.913 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.913 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.913 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.914 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache device inventory", 2023-08-25T05:00:46.914 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.914 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.914 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.914 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.915 INFO:teuthology.orchestra.run.smithi145.stdout: "device_enhanced_scan": { 2023-08-25T05:00:46.915 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_enhanced_scan", 2023-08-25T05:00:46.915 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.915 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.915 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.916 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.916 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.916 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.916 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.916 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T05:00:46.917 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.917 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.917 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.917 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.918 INFO:teuthology.orchestra.run.smithi145.stdout: "facts_cache_timeout": { 2023-08-25T05:00:46.918 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "facts_cache_timeout", 2023-08-25T05:00:46.918 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.918 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.919 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.919 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:46.919 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.920 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.920 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.920 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache host facts data", 2023-08-25T05:00:46.920 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.920 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.921 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.922 INFO:teuthology.orchestra.run.smithi145.stdout: "host_check_interval": { 2023-08-25T05:00:46.922 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "host_check_interval", 2023-08-25T05:00:46.922 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:46.923 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.923 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.924 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:46.924 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.924 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.925 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.925 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to perform a host check", 2023-08-25T05:00:46.925 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.926 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.926 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.926 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.926 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:46.927 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:46.927 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.927 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.928 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.928 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.928 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.928 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.928 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.929 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.930 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.930 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.930 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.930 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.931 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.931 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.933 INFO:teuthology.orchestra.run.smithi145.stdout: "log_refresh_metadata": { 2023-08-25T05:00:46.933 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_refresh_metadata", 2023-08-25T05:00:46.933 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.933 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.933 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.934 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.934 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.934 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.934 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.934 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T05:00:46.935 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.935 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.935 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.935 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.935 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:46.936 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:46.936 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.936 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.937 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.937 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.937 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.937 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.938 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.938 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T05:00:46.938 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.938 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.939 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.939 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.939 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:46.939 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:46.939 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.940 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.940 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.940 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:46.940 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.940 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:46.941 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:46.942 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:46.942 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.942 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.942 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.942 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.943 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.943 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:46.943 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:46.943 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.944 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.944 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:46.944 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.944 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.944 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.945 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.945 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:46.945 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.945 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.945 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.946 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.947 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.947 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.947 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.947 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.947 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T05:00:46.948 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.948 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.948 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.948 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.948 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T05:00:46.949 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T05:00:46.949 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.949 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.949 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.949 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "*", 2023-08-25T05:00:46.950 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.950 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.950 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.950 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T05:00:46.950 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.951 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.951 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.951 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.951 INFO:teuthology.orchestra.run.smithi145.stdout: "max_count_per_host": { 2023-08-25T05:00:46.951 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_count_per_host", 2023-08-25T05:00:46.952 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.952 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.952 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.952 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:46.953 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.953 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.953 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.954 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of daemons per service per host", 2023-08-25T05:00:46.954 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.954 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.955 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.955 INFO:teuthology.orchestra.run.smithi145.stdout: "max_osd_draining_count": { 2023-08-25T05:00:46.955 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_osd_draining_count", 2023-08-25T05:00:46.955 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.956 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.956 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.956 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:46.956 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.957 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.957 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.957 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T05:00:46.957 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.957 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.958 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.958 INFO:teuthology.orchestra.run.smithi145.stdout: "migration_current": { 2023-08-25T05:00:46.958 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "migration_current", 2023-08-25T05:00:46.958 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.959 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.959 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.959 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.959 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.959 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.960 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.960 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "internal - do not modify", 2023-08-25T05:00:46.960 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.960 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.960 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.961 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:46.961 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:46.961 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.961 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.962 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.962 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "root", 2023-08-25T05:00:46.962 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.962 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.962 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:46.963 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm-package", 2023-08-25T05:00:46.963 INFO:teuthology.orchestra.run.smithi145.stdout: "root" 2023-08-25T05:00:46.963 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:46.963 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T05:00:46.963 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.964 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.964 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.964 INFO:teuthology.orchestra.run.smithi145.stdout: "prometheus_alerts_path": { 2023-08-25T05:00:46.965 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus_alerts_path", 2023-08-25T05:00:46.965 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.965 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.965 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.965 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T05:00:46.966 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.966 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.966 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.966 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T05:00:46.966 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_insecure": { 2023-08-25T05:00:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_insecure", 2023-08-25T05:00:46.968 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.968 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.968 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.968 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.969 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.969 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.969 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.969 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_password": { 2023-08-25T05:00:46.970 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_password", 2023-08-25T05:00:46.971 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.971 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.971 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.971 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.972 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.972 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.972 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.972 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T05:00:46.972 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.973 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.973 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.973 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_url": { 2023-08-25T05:00:46.974 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_url", 2023-08-25T05:00:46.974 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.974 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.974 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.974 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.975 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.975 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.975 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.975 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T05:00:46.976 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.976 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.976 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.976 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_username": { 2023-08-25T05:00:46.977 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_username", 2023-08-25T05:00:46.977 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.977 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.978 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.978 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.978 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.978 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.979 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.979 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T05:00:46.979 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.979 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.980 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.980 INFO:teuthology.orchestra.run.smithi145.stdout: "secure_monitoring_stack": { 2023-08-25T05:00:46.980 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "secure_monitoring_stack", 2023-08-25T05:00:46.980 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.981 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.981 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.981 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.981 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.981 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.982 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.982 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T05:00:46.982 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.982 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.983 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.983 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.983 INFO:teuthology.orchestra.run.smithi145.stdout: "service_discovery_port": { 2023-08-25T05:00:46.983 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "service_discovery_port", 2023-08-25T05:00:46.984 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:46.984 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.984 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.985 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8765", 2023-08-25T05:00:46.985 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.985 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.986 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.986 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "cephadm service discovery port", 2023-08-25T05:00:46.986 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.986 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.987 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.987 INFO:teuthology.orchestra.run.smithi145.stdout: "ssh_config_file": { 2023-08-25T05:00:46.988 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssh_config_file", 2023-08-25T05:00:46.988 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:46.988 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.989 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.989 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:46.989 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.990 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.991 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.991 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T05:00:46.991 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.992 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.992 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.992 INFO:teuthology.orchestra.run.smithi145.stdout: "use_agent": { 2023-08-25T05:00:46.993 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_agent", 2023-08-25T05:00:46.993 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.993 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.994 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.994 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:46.994 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:46.995 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:46.995 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:46.996 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T05:00:46.996 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:46.997 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:46.997 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:46.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:46.998 INFO:teuthology.orchestra.run.smithi145.stdout: "use_repo_digest": { 2023-08-25T05:00:46.998 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_repo_digest", 2023-08-25T05:00:46.998 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:46.999 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:46.999 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:46.999 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:46.999 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.000 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.000 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.000 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T05:00:47.001 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.001 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.001 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.001 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_failed_host_check": { 2023-08-25T05:00:47.002 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_failed_host_check", 2023-08-25T05:00:47.002 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.002 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.003 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.003 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.003 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.004 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.004 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.004 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T05:00:47.005 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.005 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.005 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.006 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_daemons": { 2023-08-25T05:00:47.006 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_daemons", 2023-08-25T05:00:47.006 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.006 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.007 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.007 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.007 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.007 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.008 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.008 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T05:00:47.008 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.008 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.009 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.009 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.009 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_hosts": { 2023-08-25T05:00:47.009 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_hosts", 2023-08-25T05:00:47.009 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.010 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.010 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.010 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.010 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.010 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.011 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.011 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T05:00:47.011 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.011 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.011 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.012 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.012 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.012 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.012 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.012 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crash", 2023-08-25T05:00:47.013 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.013 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.013 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.013 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.013 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.014 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.014 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.014 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.014 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.014 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.015 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.015 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.015 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.015 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.015 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.016 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.016 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.016 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.016 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.017 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.017 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.017 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.017 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.018 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.018 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.019 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.019 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.019 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.019 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.019 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.020 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.020 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.020 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.020 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.021 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.021 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.021 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.021 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.022 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.022 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.022 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.022 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.023 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.023 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.023 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.023 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.023 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.024 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.024 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.024 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.024 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.024 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.025 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.025 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.025 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.025 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.025 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.026 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.026 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.026 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.026 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.026 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.027 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.027 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.027 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.027 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.028 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.028 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.028 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.028 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.028 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: "retain_interval": { 2023-08-25T05:00:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retain_interval", 2023-08-25T05:00:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.029 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.030 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "31536000", 2023-08-25T05:00:47.030 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.030 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.030 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.031 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T05:00:47.031 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.031 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.031 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.032 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.032 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_recent_interval": { 2023-08-25T05:00:47.032 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_recent_interval", 2023-08-25T05:00:47.033 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.033 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.033 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.033 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1209600", 2023-08-25T05:00:47.034 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.034 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.034 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.035 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T05:00:47.035 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.035 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.036 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.036 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.036 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.037 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.037 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dashboard", 2023-08-25T05:00:47.037 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.038 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.038 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T05:00:47.039 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T05:00:47.039 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.039 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.040 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.040 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:47.040 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.041 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.041 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.041 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.042 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.042 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.042 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.043 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T05:00:47.043 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T05:00:47.044 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.044 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.044 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.044 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.045 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.045 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.045 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.045 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.046 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.046 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.047 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.048 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T05:00:47.048 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T05:00:47.049 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.049 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.050 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.050 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.050 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.051 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.051 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.051 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.051 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.051 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.052 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.052 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_ENABLED": { 2023-08-25T05:00:47.052 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T05:00:47.053 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.053 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.053 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.053 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.053 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.055 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.055 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T05:00:47.055 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T05:00:47.055 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.056 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.056 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.056 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.056 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.057 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.057 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.057 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.057 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.058 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.058 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.058 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.058 INFO:teuthology.orchestra.run.smithi145.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T05:00:47.058 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T05:00:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.059 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.060 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.060 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.060 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.060 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.061 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.061 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.061 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T05:00:47.062 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T05:00:47.062 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.062 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.062 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.063 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.064 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.064 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.064 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.064 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T05:00:47.065 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T05:00:47.065 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.065 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.065 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.066 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.066 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.066 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.066 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.066 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.067 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.067 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.067 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.068 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T05:00:47.068 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T05:00:47.068 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.068 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.069 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.069 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.069 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.070 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.070 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.070 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.071 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.071 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.071 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.072 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T05:00:47.072 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T05:00:47.072 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.072 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.073 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.073 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.073 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.073 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.074 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.074 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.074 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.074 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.075 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.075 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.075 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T05:00:47.075 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T05:00:47.076 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.076 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.076 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.076 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.077 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.077 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.077 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.077 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.077 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.078 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.078 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.078 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T05:00:47.078 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T05:00:47.079 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.079 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.079 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.079 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.079 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.080 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.080 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.080 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.080 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.081 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.081 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.081 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T05:00:47.081 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T05:00:47.082 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.082 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.082 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.082 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.083 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.083 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.083 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.083 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.084 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.084 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.084 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.084 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.085 INFO:teuthology.orchestra.run.smithi145.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T05:00:47.085 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T05:00:47.085 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.086 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.086 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.086 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.086 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.086 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.087 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.087 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.087 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.087 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.088 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.088 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T05:00:47.088 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T05:00:47.088 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.089 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.089 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.089 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:47.089 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.090 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.090 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.090 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.090 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.091 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.091 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.091 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T05:00:47.092 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T05:00:47.092 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.093 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.093 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.093 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.093 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.094 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.094 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.094 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.094 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.095 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.095 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.095 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.096 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_URL": { 2023-08-25T05:00:47.096 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_URL", 2023-08-25T05:00:47.096 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.096 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.097 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.097 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.097 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.098 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.098 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.098 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.098 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.098 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.099 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.099 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T05:00:47.099 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T05:00:47.100 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.100 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.100 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.100 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:47.100 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.101 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.101 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.101 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.102 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.102 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.102 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.102 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.102 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T05:00:47.103 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T05:00:47.103 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.104 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.104 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.104 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.105 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.105 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.105 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.106 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.106 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.106 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.107 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.107 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.108 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T05:00:47.108 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T05:00:47.108 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.108 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.109 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.109 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.109 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.110 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.110 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.110 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.111 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.111 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.111 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.112 INFO:teuthology.orchestra.run.smithi145.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T05:00:47.113 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T05:00:47.113 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.113 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.114 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.114 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.115 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.115 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.116 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.116 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.117 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.118 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.118 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.119 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.119 INFO:teuthology.orchestra.run.smithi145.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T05:00:47.119 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T05:00:47.120 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.121 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.121 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.121 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.122 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.122 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.122 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.124 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.124 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.124 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.125 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.125 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.125 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T05:00:47.125 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T05:00:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.126 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.127 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.127 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.127 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.127 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.128 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.128 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.128 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.128 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T05:00:47.128 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T05:00:47.129 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.129 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.129 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.129 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.129 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.130 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.130 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.130 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.130 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.131 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.131 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.131 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T05:00:47.132 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T05:00:47.132 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.132 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.132 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.133 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.133 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.133 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.133 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.134 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.134 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.134 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.134 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.135 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.135 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T05:00:47.135 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T05:00:47.135 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.136 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.136 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.136 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.137 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.137 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.137 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.137 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.138 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.138 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.138 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.138 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.139 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T05:00:47.139 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T05:00:47.139 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.140 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.140 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.140 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.141 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.141 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.141 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.141 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.142 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.142 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.143 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.143 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T05:00:47.143 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T05:00:47.144 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.144 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.145 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.145 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.145 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.146 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.146 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.147 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.147 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.147 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.148 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.148 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T05:00:47.149 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T05:00:47.149 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.150 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.150 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.151 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.151 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.151 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.152 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.152 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.152 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.152 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.153 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.153 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T05:00:47.154 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T05:00:47.154 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.154 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.154 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.155 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.155 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.155 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.155 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.156 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.156 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.156 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.156 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.157 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T05:00:47.157 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T05:00:47.157 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.157 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.158 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.158 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.158 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.158 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.159 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.159 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.159 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.159 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.159 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.160 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T05:00:47.160 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T05:00:47.160 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.160 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.161 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.161 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.161 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.161 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.161 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.163 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T05:00:47.163 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T05:00:47.163 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.163 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.163 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.164 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T05:00:47.164 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.164 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.164 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.164 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.165 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.165 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.165 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.166 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T05:00:47.166 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T05:00:47.166 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.167 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.167 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.167 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:47.167 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.167 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.168 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.168 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.168 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.168 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.169 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.169 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T05:00:47.169 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T05:00:47.170 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.170 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.170 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.170 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8", 2023-08-25T05:00:47.170 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.171 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.171 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.171 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.171 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.172 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.172 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.172 INFO:teuthology.orchestra.run.smithi145.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T05:00:47.173 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T05:00:47.173 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.173 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.173 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.174 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "45", 2023-08-25T05:00:47.174 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.174 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.174 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.174 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.175 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.175 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.175 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.176 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T05:00:47.176 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T05:00:47.176 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.177 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.177 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.177 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.177 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.178 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.178 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.178 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.178 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.179 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.179 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.180 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T05:00:47.180 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T05:00:47.181 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.181 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.181 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.182 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:47.182 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.182 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.182 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.183 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.183 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.183 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.183 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.184 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T05:00:47.184 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T05:00:47.184 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.184 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.185 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.185 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.185 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.185 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.186 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.186 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.186 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.186 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.186 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.187 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.187 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T05:00:47.187 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T05:00:47.187 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.187 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.188 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.188 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.188 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.188 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.188 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.189 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.189 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.189 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.189 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T05:00:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T05:00:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.191 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:47.191 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.191 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.191 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.192 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.192 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.192 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.192 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.192 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.193 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T05:00:47.194 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T05:00:47.194 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.194 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.194 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.195 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:47.195 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.195 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.195 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.195 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.196 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.196 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.196 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.196 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.196 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T05:00:47.197 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T05:00:47.197 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.197 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.197 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.198 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:47.198 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.198 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.198 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.198 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.199 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.199 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.200 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.200 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.200 INFO:teuthology.orchestra.run.smithi145.stdout: "cross_origin_url": { 2023-08-25T05:00:47.200 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cross_origin_url", 2023-08-25T05:00:47.201 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.201 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.202 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.202 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.202 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.203 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.203 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.203 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.204 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.204 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.205 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.205 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.205 INFO:teuthology.orchestra.run.smithi145.stdout: "crt_file": { 2023-08-25T05:00:47.206 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crt_file", 2023-08-25T05:00:47.206 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.206 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.207 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.207 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.207 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.207 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.208 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.208 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.208 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.208 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.208 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.209 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.209 INFO:teuthology.orchestra.run.smithi145.stdout: "debug": { 2023-08-25T05:00:47.209 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "debug", 2023-08-25T05:00:47.209 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.209 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.210 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.210 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.210 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.210 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.210 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.211 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable/disable debug options", 2023-08-25T05:00:47.211 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.211 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.211 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.211 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.212 INFO:teuthology.orchestra.run.smithi145.stdout: "jwt_token_ttl": { 2023-08-25T05:00:47.212 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "jwt_token_ttl", 2023-08-25T05:00:47.212 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.212 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.213 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.213 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "28800", 2023-08-25T05:00:47.213 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.213 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.213 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.214 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.214 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.214 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.214 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.214 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.215 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:47.215 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:47.215 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.215 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.215 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.216 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.217 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.217 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.217 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.218 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.218 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.218 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.218 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.219 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.219 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.219 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.219 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.221 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.222 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.222 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.225 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.226 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.227 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.228 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.229 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.229 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.229 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.230 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.230 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.230 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.230 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.230 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.231 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.231 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.231 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.233 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.233 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.243 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.244 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.244 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.245 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.245 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.245 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.245 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.246 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.247 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.247 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.247 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.247 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.248 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.248 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.248 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.248 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.249 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.249 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.249 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.249 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.249 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.250 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.250 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.250 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.255 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.256 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.256 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.257 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.257 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.257 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.258 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.258 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.258 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.258 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.259 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.259 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.259 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.260 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.260 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.260 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.260 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.261 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.261 INFO:teuthology.orchestra.run.smithi145.stdout: "motd": { 2023-08-25T05:00:47.261 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "motd", 2023-08-25T05:00:47.261 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.262 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.262 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.262 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.262 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.263 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.263 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.263 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "The message of the day", 2023-08-25T05:00:47.264 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.264 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.264 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.264 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.264 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect_resolve_ip_addr": { 2023-08-25T05:00:47.265 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T05:00:47.265 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.265 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.265 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.266 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.268 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.268 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.268 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.268 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.269 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.270 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:47.271 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.271 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.271 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.271 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.271 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.272 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.272 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.272 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.272 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:47.272 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:47.273 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.273 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.273 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.273 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8080", 2023-08-25T05:00:47.274 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.274 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.274 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.275 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.275 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.275 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.275 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.275 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.276 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:47.276 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:47.276 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.276 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.276 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.277 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.277 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.277 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.278 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.278 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.279 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.279 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.279 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.279 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.279 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl_server_port": { 2023-08-25T05:00:47.280 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl_server_port", 2023-08-25T05:00:47.280 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.280 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.280 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.281 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8443", 2023-08-25T05:00:47.281 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.281 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.281 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.282 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.282 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.282 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.282 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.282 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.283 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:47.283 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:47.283 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.283 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.283 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.284 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "redirect", 2023-08-25T05:00:47.284 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.285 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.285 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.285 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.285 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect" 2023-08-25T05:00:47.286 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.286 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.286 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.287 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.287 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.287 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.287 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:47.287 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:47.288 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.289 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.289 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.289 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:47.289 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:47.289 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:47.290 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.290 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.290 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.290 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.291 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.291 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.291 INFO:teuthology.orchestra.run.smithi145.stdout: "url_prefix": { 2023-08-25T05:00:47.291 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url_prefix", 2023-08-25T05:00:47.292 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.292 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.292 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.292 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.293 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.293 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.293 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.293 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.294 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.294 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.294 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.294 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.295 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.295 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.295 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.296 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "devicehealth", 2023-08-25T05:00:47.296 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.296 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.296 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.297 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_monitoring": { 2023-08-25T05:00:47.297 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_monitoring", 2023-08-25T05:00:47.297 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.297 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.298 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.298 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.299 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.299 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.299 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.300 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "monitor device health metrics", 2023-08-25T05:00:47.300 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.300 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.300 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.301 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.301 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.302 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.302 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.303 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.303 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.303 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.303 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.304 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.304 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.304 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.304 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.305 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.305 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.305 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.305 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.306 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.306 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.306 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.307 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.307 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.307 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.308 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.309 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.309 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.310 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.310 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.310 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.310 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.311 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.311 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.311 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.311 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.312 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.313 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.313 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.313 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.314 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.314 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.315 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.315 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.315 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.316 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.316 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.316 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.317 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.317 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.318 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.318 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.318 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.319 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.319 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.320 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.320 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.320 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.322 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.323 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.323 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.323 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.323 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.324 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.324 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.324 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.324 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.324 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.325 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.325 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.325 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.325 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.325 INFO:teuthology.orchestra.run.smithi145.stdout: "mark_out_threshold": { 2023-08-25T05:00:47.326 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mark_out_threshold", 2023-08-25T05:00:47.326 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.326 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.326 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.326 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2419200", 2023-08-25T05:00:47.327 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.327 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.327 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.327 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T05:00:47.327 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.328 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.328 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.328 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.329 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_name": { 2023-08-25T05:00:47.329 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_name", 2023-08-25T05:00:47.329 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.329 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.329 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.330 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "device_health_metrics", 2023-08-25T05:00:47.330 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.330 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.330 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.330 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T05:00:47.331 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.331 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.331 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.331 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.331 INFO:teuthology.orchestra.run.smithi145.stdout: "retention_period": { 2023-08-25T05:00:47.332 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retention_period", 2023-08-25T05:00:47.332 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.332 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.332 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.332 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15552000", 2023-08-25T05:00:47.333 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.333 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.333 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.334 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain device health metrics", 2023-08-25T05:00:47.334 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.334 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.334 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.335 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_frequency": { 2023-08-25T05:00:47.335 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_frequency", 2023-08-25T05:00:47.335 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.335 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.335 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.336 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:47.336 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.336 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.336 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.336 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T05:00:47.337 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.337 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.337 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.337 INFO:teuthology.orchestra.run.smithi145.stdout: "self_heal": { 2023-08-25T05:00:47.338 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "self_heal", 2023-08-25T05:00:47.338 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.338 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.338 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.338 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.339 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.339 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.339 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.339 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T05:00:47.340 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.340 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.340 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.340 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.341 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:47.341 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:47.341 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.341 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.342 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.342 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:47.342 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.342 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.343 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.343 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T05:00:47.343 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.343 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.344 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.344 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_threshold": { 2023-08-25T05:00:47.344 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_threshold", 2023-08-25T05:00:47.345 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.345 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.345 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.345 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7257600", 2023-08-25T05:00:47.346 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.346 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.347 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.347 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T05:00:47.347 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.347 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.348 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.348 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.348 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.349 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.349 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "diskprediction_local", 2023-08-25T05:00:47.349 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.349 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.350 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.350 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.350 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.351 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.351 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.352 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.352 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.352 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.353 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi145.stdout: "predict_interval": { 2023-08-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predict_interval", 2023-08-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi145.stdout: "predictor_model": { 2023-08-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predictor_model", 2023-08-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "prophetstor", 2023-08-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.386 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.386 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.386 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.386 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:47.386 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "influx", 2023-08-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": false, 2023-08-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "influxdb python module not found", 2023-08-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi145.stdout: "batch_size": { 2023-08-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "batch_size", 2023-08-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5000", 2023-08-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi145.stdout: "database": { 2023-08-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "database", 2023-08-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "ceph", 2023-08-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": { 2023-08-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "hostname", 2023-08-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server hostname", 2023-08-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "30", 2023-08-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "5", 2023-08-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.417 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.417 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.417 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.417 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.418 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.418 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.418 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.419 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.419 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.419 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.420 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.420 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.420 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.420 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.421 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.421 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.421 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.421 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.422 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.422 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.422 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.422 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.422 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.423 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.423 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.424 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.424 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.424 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.424 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.425 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.425 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.425 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.425 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.425 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.426 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.426 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.426 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.426 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.427 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.427 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.427 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.427 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.427 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: "password": { 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "password", 2023-08-25T05:00:47.428 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.429 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.429 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.429 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.429 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.429 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.430 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.430 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "password of InfluxDB server user", 2023-08-25T05:00:47.430 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.430 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.430 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.431 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.431 INFO:teuthology.orchestra.run.smithi145.stdout: "port": { 2023-08-25T05:00:47.431 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "port", 2023-08-25T05:00:47.431 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.431 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.432 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.432 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8086", 2023-08-25T05:00:47.432 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.432 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.433 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.433 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server port", 2023-08-25T05:00:47.433 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.433 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.434 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.434 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:47.434 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:47.435 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.435 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.435 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.435 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "false", 2023-08-25T05:00:47.436 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.436 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.436 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.436 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:47.436 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.437 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.437 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.437 INFO:teuthology.orchestra.run.smithi145.stdout: "threads": { 2023-08-25T05:00:47.437 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threads", 2023-08-25T05:00:47.438 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.438 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.438 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.438 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:47.438 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "32", 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.439 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.440 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.440 INFO:teuthology.orchestra.run.smithi145.stdout: "username": { 2023-08-25T05:00:47.440 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "username", 2023-08-25T05:00:47.440 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.440 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.441 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.441 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.441 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.441 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.441 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.442 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "username of InfluxDB server user", 2023-08-25T05:00:47.442 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.442 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.442 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "verify_ssl": { 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "verify_ssl", 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.443 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "true", 2023-08-25T05:00:47.444 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.444 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.444 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.444 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:47.444 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.445 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.445 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.445 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.445 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.446 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.446 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "insights", 2023-08-25T05:00:47.446 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.446 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.446 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.447 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.448 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.448 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.448 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.448 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.448 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.449 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.449 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.449 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.449 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.449 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.450 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.450 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.450 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.450 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.451 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.451 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.451 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.451 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.451 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.452 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.452 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.452 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.452 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.452 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.453 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.453 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.453 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.453 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.454 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.454 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.454 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.454 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.455 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.455 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.455 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.455 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.456 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.456 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.456 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.456 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.457 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.457 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.457 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.457 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.457 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.458 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.458 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.458 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.459 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.459 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.459 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.459 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.460 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.460 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.460 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.460 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.461 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.461 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.461 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.461 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.461 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.462 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.462 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.462 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "iostat", 2023-08-25T05:00:47.462 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.463 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.464 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.464 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.464 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.465 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.465 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.465 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.465 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.466 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.466 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.466 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.466 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.466 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.467 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.468 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.468 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.468 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.468 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.468 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.469 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.469 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.469 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.469 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.469 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.470 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.470 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.470 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.470 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.471 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.471 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.471 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.471 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.471 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.472 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.472 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.472 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.472 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.472 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.473 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.473 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.473 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.473 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.473 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.474 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.474 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.474 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.474 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.474 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.475 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.476 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.476 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.476 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.476 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.476 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.477 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.477 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.477 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.477 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.477 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.478 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "k8sevents", 2023-08-25T05:00:47.478 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.478 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.478 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.478 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_event_retention_days": { 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ceph_event_retention_days", 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7", 2023-08-25T05:00:47.479 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.480 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.480 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.480 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T05:00:47.480 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.480 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.481 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.481 INFO:teuthology.orchestra.run.smithi145.stdout: "config_check_secs": { 2023-08-25T05:00:47.481 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_check_secs", 2023-08-25T05:00:47.481 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.482 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.482 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.482 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:47.483 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "10", 2023-08-25T05:00:47.483 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.483 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.483 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T05:00:47.483 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.484 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.485 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.485 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.485 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.485 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.485 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.486 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.486 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.486 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.486 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.486 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.487 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.487 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.487 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.487 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.487 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.488 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.488 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.488 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.488 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.488 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.489 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.489 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.489 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.489 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.489 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.490 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.490 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.490 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.490 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.490 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.491 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.491 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.491 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.491 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.492 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.492 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.492 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.492 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.492 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.493 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.493 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.493 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.493 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.494 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.495 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.495 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.495 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.495 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.495 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.496 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.496 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.496 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.496 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.496 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.497 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.497 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.497 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.497 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.497 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.498 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.498 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.498 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.498 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "localpool", 2023-08-25T05:00:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: "failure_domain": { 2023-08-25T05:00:47.499 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "failure_domain", 2023-08-25T05:00:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "host", 2023-08-25T05:00:47.500 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "failure domain for any created local pool", 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.501 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.502 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.502 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.502 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.503 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.503 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.503 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.503 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.503 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.504 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.504 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.504 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.504 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.505 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.505 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.505 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.505 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.505 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.506 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.507 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.507 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.507 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.507 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.507 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.508 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.508 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.508 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.508 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.508 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.509 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.509 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.509 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.509 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.509 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.510 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.510 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.510 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.510 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.510 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.511 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.512 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.512 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.512 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.512 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.512 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.513 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.513 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.513 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.513 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.514 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.514 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.514 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.514 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.514 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.515 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.515 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.515 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.515 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.515 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.516 INFO:teuthology.orchestra.run.smithi145.stdout: "min_size": { 2023-08-25T05:00:47.516 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_size", 2023-08-25T05:00:47.516 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.516 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.516 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.517 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.517 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.517 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.517 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.517 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default min_size for any created local pool", 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: "num_rep": { 2023-08-25T05:00:47.518 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "num_rep", 2023-08-25T05:00:47.519 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.519 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.519 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.519 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3", 2023-08-25T05:00:47.519 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.520 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.520 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.520 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default replica count for any created local pool", 2023-08-25T05:00:47.520 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.520 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.521 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.521 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.521 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_num": { 2023-08-25T05:00:47.521 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_num", 2023-08-25T05:00:47.521 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.522 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.522 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.522 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "128", 2023-08-25T05:00:47.522 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.522 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.523 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.523 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T05:00:47.523 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.523 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.524 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.525 INFO:teuthology.orchestra.run.smithi145.stdout: "prefix": { 2023-08-25T05:00:47.525 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prefix", 2023-08-25T05:00:47.526 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.526 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.526 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.526 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.526 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.527 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.527 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.527 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name prefix for any created local pool", 2023-08-25T05:00:47.527 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.528 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.528 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.528 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.528 INFO:teuthology.orchestra.run.smithi145.stdout: "subtree": { 2023-08-25T05:00:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "subtree", 2023-08-25T05:00:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "rack", 2023-08-25T05:00:47.530 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.530 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.530 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.531 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T05:00:47.531 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T05:00:47.531 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.532 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.533 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.533 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mds_autoscaler", 2023-08-25T05:00:47.533 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.533 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.534 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.534 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.534 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.534 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.535 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.536 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.536 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.536 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.536 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.536 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.537 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.537 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.537 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.538 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.538 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.538 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.538 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.538 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.539 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.539 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.539 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.539 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.540 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.540 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.540 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.540 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.541 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.541 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.541 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.541 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.541 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.542 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.542 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.542 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.542 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.543 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.543 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.543 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.543 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.544 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.544 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.544 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.544 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.545 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.545 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.545 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.545 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.545 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.546 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.546 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.546 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.546 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.547 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.547 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.547 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.547 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.548 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.548 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.548 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.549 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.550 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.550 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.550 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.550 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.550 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.551 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.551 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.552 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirroring", 2023-08-25T05:00:47.552 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.552 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.552 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.553 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.553 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.553 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.553 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.553 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.554 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.554 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.554 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.554 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.555 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.555 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.555 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.555 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.556 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.556 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.556 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.557 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.557 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.557 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.557 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.558 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.558 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.558 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.558 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.559 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.559 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.559 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.559 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.560 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.560 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.560 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.560 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.560 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.561 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.561 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.561 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.561 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.562 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.562 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.562 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.563 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.563 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.563 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.563 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.564 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.564 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.564 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.564 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.565 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.565 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.565 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.566 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.567 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.567 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.568 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.568 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.568 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.569 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.569 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.569 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.569 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.570 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.570 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.570 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.570 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.571 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.571 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.571 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.572 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.572 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.572 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.573 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "nfs", 2023-08-25T05:00:47.573 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.573 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.573 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.574 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.574 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.575 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.575 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.575 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.576 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.576 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.576 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.576 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.577 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.577 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.577 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.578 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.578 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.578 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.578 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.578 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.579 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.579 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.580 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.580 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.580 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.581 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.581 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.581 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.581 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.582 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.582 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.582 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.582 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.583 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.583 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.583 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.583 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.584 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.584 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.584 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.584 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.585 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.585 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.585 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.586 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.586 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.586 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.586 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.587 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.587 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.587 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.587 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.587 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.588 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.588 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.588 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.588 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.588 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.589 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.589 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.589 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.589 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.589 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.590 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.590 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.590 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.590 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.590 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.591 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.591 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.591 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.591 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.591 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.592 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.592 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.592 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.592 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:47.593 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.593 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.593 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.593 INFO:teuthology.orchestra.run.smithi145.stdout: "fail_fs": { 2023-08-25T05:00:47.593 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "fail_fs", 2023-08-25T05:00:47.594 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.594 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.594 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.594 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.594 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.595 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.595 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.595 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T05:00:47.595 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.595 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.596 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.596 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.596 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.596 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.597 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.597 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.597 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.597 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.597 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.598 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.598 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.598 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.598 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.599 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.599 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.599 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.599 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.600 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.600 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.600 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.600 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.601 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.601 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.601 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.601 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.601 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.602 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.602 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.602 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.602 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.602 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.603 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.603 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.603 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.603 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.603 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.604 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.604 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.604 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.604 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.605 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.605 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.605 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.606 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.606 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.606 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.607 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.607 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.607 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.608 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.608 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.608 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.608 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.609 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.609 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.609 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.609 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.610 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.610 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.611 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.611 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.611 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.611 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.612 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.612 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.612 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.613 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.613 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.613 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.613 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.614 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.614 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator": { 2023-08-25T05:00:47.615 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:47.615 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.615 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.616 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.616 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.616 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.616 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.617 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.617 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm", 2023-08-25T05:00:47.617 INFO:teuthology.orchestra.run.smithi145.stdout: "rook", 2023-08-25T05:00:47.617 INFO:teuthology.orchestra.run.smithi145.stdout: "test_orchestrator" 2023-08-25T05:00:47.617 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.618 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Orchestrator backend", 2023-08-25T05:00:47.618 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.618 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.618 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.619 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.619 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.620 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.620 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.620 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_perf_query", 2023-08-25T05:00:47.620 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.620 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.621 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.622 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.622 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.622 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.622 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.622 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.623 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.623 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.623 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.623 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.623 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.624 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.624 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.624 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.625 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.626 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.626 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.626 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.626 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.626 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.627 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.627 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.627 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.627 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.628 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.628 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.635 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.635 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.635 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.636 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.636 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.637 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.637 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.637 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.637 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.637 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.638 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.638 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.638 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.638 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.638 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.639 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.639 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.639 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.639 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.639 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.641 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.642 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.642 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.642 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.642 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.642 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.643 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.643 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.643 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.643 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.643 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.644 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.644 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.644 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_support", 2023-08-25T05:00:47.644 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.644 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.645 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.646 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.646 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.646 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.646 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.647 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.647 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.647 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.647 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.648 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.649 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.649 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.649 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.650 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.650 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.650 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.651 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.651 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.651 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.652 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.652 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.652 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.653 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.653 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.653 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.653 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.653 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.654 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.654 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.655 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.655 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.655 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.656 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.657 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.657 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.657 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.658 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.658 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.658 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.659 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.659 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.659 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.660 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.660 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.660 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.660 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.661 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.661 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.662 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.662 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.662 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.662 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.663 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.663 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.663 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.663 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.664 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.664 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.664 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.664 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.664 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.665 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.665 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.666 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.666 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.667 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.667 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_autoscaler", 2023-08-25T05:00:47.667 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.668 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.668 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.668 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.669 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.669 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.669 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.669 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.670 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.670 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.670 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.670 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.670 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.671 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.671 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.671 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.671 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.672 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.672 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.672 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.673 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.673 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.673 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.674 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.674 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.674 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.674 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.675 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.675 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.675 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.675 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.676 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.676 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.676 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.677 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.677 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.677 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.678 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.679 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.679 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.679 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.679 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.680 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.680 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.680 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.680 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.680 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.681 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.681 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.681 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.682 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.682 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.682 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.682 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.683 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.683 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.683 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.683 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.684 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.684 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.684 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.684 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.685 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.685 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.685 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.685 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.685 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.686 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.686 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.686 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.686 INFO:teuthology.orchestra.run.smithi145.stdout: "noautoscale": { 2023-08-25T05:00:47.686 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "noautoscale", 2023-08-25T05:00:47.687 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.687 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.687 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.687 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.687 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.688 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.688 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.688 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "global autoscale flag", 2023-08-25T05:00:47.688 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T05:00:47.692 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.692 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.693 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:47.693 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:47.693 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.693 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.694 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.694 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:47.694 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.694 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.694 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.695 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.695 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.695 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.695 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.696 INFO:teuthology.orchestra.run.smithi145.stdout: "threshold": { 2023-08-25T05:00:47.696 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threshold", 2023-08-25T05:00:47.696 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:47.696 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.696 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.697 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:47.698 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1.0", 2023-08-25T05:00:47.698 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.698 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.698 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "scaling threshold", 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.700 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.700 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "progress", 2023-08-25T05:00:47.700 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.700 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.700 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.701 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_pg_recovery_event": { 2023-08-25T05:00:47.701 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_pg_recovery_event", 2023-08-25T05:00:47.701 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.701 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.701 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.702 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.702 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.702 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.702 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.702 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T05:00:47.703 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.703 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.703 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.703 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:47.704 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:47.704 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.704 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.704 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.705 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.706 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.706 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.706 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.706 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.707 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.707 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.707 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.707 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.707 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.708 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.708 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.708 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.708 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.709 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.710 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.710 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.710 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.710 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.710 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.711 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.711 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.711 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.711 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.711 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.712 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.712 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.712 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.712 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.712 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.713 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.713 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.713 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.713 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.713 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.714 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.715 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.715 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.715 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.715 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.715 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.716 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.716 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.716 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.716 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.716 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.717 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.717 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.717 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.717 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.718 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.718 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.718 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.718 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.718 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.719 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.719 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.719 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.719 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.719 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.720 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.720 INFO:teuthology.orchestra.run.smithi145.stdout: "max_completed_events": { 2023-08-25T05:00:47.720 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_completed_events", 2023-08-25T05:00:47.720 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.721 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.721 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.721 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "50", 2023-08-25T05:00:47.721 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.721 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.722 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.722 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "number of past completed events to remember", 2023-08-25T05:00:47.722 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.722 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.722 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.723 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long the module is going to sleep", 2023-08-25T05:00:47.725 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.725 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.725 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.725 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.725 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.726 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.726 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus", 2023-08-25T05:00:47.726 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.726 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.727 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.727 INFO:teuthology.orchestra.run.smithi145.stdout: "cache": { 2023-08-25T05:00:47.727 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cache", 2023-08-25T05:00:47.727 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.727 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.728 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.728 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.728 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.728 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.728 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.729 INFO:teuthology.orchestra.run.smithi145.stdout: "exclude_perf_counters": { 2023-08-25T05:00:47.730 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "exclude_perf_counters", 2023-08-25T05:00:47.730 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.730 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.730 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.730 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.731 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.731 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.731 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.731 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T05:00:47.731 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T05:00:47.732 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.732 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.732 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.732 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.733 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.733 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.733 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.733 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.733 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.734 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.735 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.735 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.735 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.735 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.735 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.736 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.736 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.736 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.736 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.737 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.737 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.737 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.737 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.737 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.738 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.738 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.738 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.738 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.738 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.739 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.740 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.740 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.740 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.740 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.741 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.741 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.741 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.741 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.741 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.742 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.743 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.744 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.745 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.746 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools": { 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools", 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.747 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.748 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T05:00:47.749 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T05:00:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300", 2023-08-25T05:00:47.750 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.751 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_interval": { 2023-08-25T05:00:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_interval", 2023-08-25T05:00:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:47.752 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15.0", 2023-08-25T05:00:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.753 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.754 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.755 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.756 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:47.757 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "9283", 2023-08-25T05:00:47.758 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T05:00:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.759 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "stale_cache_strategy": { 2023-08-25T05:00:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stale_cache_strategy", 2023-08-25T05:00:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "log", 2023-08-25T05:00:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.761 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.763 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "default", 2023-08-25T05:00:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.764 INFO:teuthology.orchestra.run.smithi145.stdout: "default", 2023-08-25T05:00:47.765 INFO:teuthology.orchestra.run.smithi145.stdout: "error" 2023-08-25T05:00:47.765 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.765 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.765 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.765 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.766 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.767 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.767 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:47.767 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:47.767 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:47.767 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.768 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.768 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.768 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.768 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.768 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.769 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.769 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.769 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_support", 2023-08-25T05:00:47.769 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.770 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.771 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.771 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.771 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.771 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.771 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.772 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.772 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.772 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.772 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.772 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.773 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.773 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.773 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.773 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.773 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.774 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.775 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.775 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.775 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.775 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.775 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.776 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.776 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.776 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.776 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.776 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.777 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.777 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.777 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.777 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.777 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.778 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.778 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.778 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.778 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.778 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.779 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.779 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.779 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.779 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.779 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.781 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.781 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.781 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.781 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.782 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.782 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.782 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.782 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.783 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.783 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.783 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.783 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.783 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.784 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_snap_create": { 2023-08-25T05:00:47.784 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_snap_create", 2023-08-25T05:00:47.784 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.784 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.785 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.786 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.786 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.786 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.786 INFO:teuthology.orchestra.run.smithi145.stdout: "mirror_snapshot_schedule": { 2023-08-25T05:00:47.787 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T05:00:47.787 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.787 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.787 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.787 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.788 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.788 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.788 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.788 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.788 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.789 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.789 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.789 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.789 INFO:teuthology.orchestra.run.smithi145.stdout: "trash_purge_schedule": { 2023-08-25T05:00:47.789 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "trash_purge_schedule", 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.790 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.791 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.791 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.791 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.791 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.791 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.792 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.792 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.792 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.792 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "restful", 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_auth": { 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_auth", 2023-08-25T05:00:47.793 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.794 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.794 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.794 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.794 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.794 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.795 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.795 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.795 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.795 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.795 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.796 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:47.796 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:47.796 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.796 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.797 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.798 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.798 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.798 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.798 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.799 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.799 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.799 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.799 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.799 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.800 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.800 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.800 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.800 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.800 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.801 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.801 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.801 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.801 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.801 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.802 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.802 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.802 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.802 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.803 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.804 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.804 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.804 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.804 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.804 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.805 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.805 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.805 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.805 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.806 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.806 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.806 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.806 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.806 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.807 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.808 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.808 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.808 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.808 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.808 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.809 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.809 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.809 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.809 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.809 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.810 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.810 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.810 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.811 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.811 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.811 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.811 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.811 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.812 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.812 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.812 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.812 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.812 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:47.813 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:47.813 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.813 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.813 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.813 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.814 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.815 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.815 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.815 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:47.815 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:47.816 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.816 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.816 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.816 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.817 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.817 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.817 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.817 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.817 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.818 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.818 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.818 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.818 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rgw", 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.819 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.820 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.820 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.820 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.820 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.820 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.821 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.821 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.821 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.821 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.821 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.822 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.822 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.822 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.822 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.822 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.823 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.823 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.823 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.823 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.823 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.824 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.825 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.825 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.825 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.825 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.825 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.826 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.826 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.826 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.826 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.826 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.827 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.827 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.827 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.827 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.827 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.828 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.828 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.828 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.828 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.828 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.829 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.829 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.829 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.829 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.829 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.830 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.831 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.831 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.831 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.831 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.831 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.832 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.832 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.832 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.832 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.832 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.833 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rook", 2023-08-25T05:00:47.834 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.834 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.834 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.834 INFO:teuthology.orchestra.run.smithi145.stdout: "drive_group_interval": { 2023-08-25T05:00:47.834 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "drive_group_interval", 2023-08-25T05:00:47.835 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:47.835 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.835 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.835 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300.0", 2023-08-25T05:00:47.835 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.836 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.836 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.836 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T05:00:47.836 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.836 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.837 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.837 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.837 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.837 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.837 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.839 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.839 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.839 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.839 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.839 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.840 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.840 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.840 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.840 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.840 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.841 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.841 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.841 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.841 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.841 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.842 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.843 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.843 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.843 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.843 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.843 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.844 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.844 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.844 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.844 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.844 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.845 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.845 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.845 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.845 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.845 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.846 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.846 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.846 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.846 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.846 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.847 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.847 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.847 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.847 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.847 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.848 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.848 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.848 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.848 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.848 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.849 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.849 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.849 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.849 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.849 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.850 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.850 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.850 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.850 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.850 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.851 INFO:teuthology.orchestra.run.smithi145.stdout: "storage_class": { 2023-08-25T05:00:47.851 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "storage_class", 2023-08-25T05:00:47.851 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.851 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.851 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.852 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "local", 2023-08-25T05:00:47.852 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.852 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.852 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.856 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T05:00:47.856 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.857 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.857 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.857 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.858 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.858 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.858 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.858 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "selftest", 2023-08-25T05:00:47.858 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.859 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.859 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.859 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.859 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.860 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.860 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.860 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.860 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.860 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.861 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.861 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.861 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.861 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.861 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.862 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.862 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.863 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.863 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.863 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.863 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.863 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.864 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.864 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.864 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.865 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.865 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.865 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.865 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.865 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.866 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.866 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.866 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.866 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.866 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.867 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.868 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.869 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.869 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.872 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.873 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.873 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.873 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.873 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.874 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.874 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.874 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.874 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.874 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.875 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.875 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.875 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.875 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.875 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.876 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.876 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.876 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.876 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.876 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.877 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.877 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.877 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.877 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.877 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.878 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.878 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.879 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.879 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.879 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.879 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.880 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.880 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.881 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.881 INFO:teuthology.orchestra.run.smithi145.stdout: "roption1": { 2023-08-25T05:00:47.882 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption1", 2023-08-25T05:00:47.882 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.882 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.882 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.882 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.883 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.883 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.883 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.883 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.883 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.884 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.884 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.884 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.884 INFO:teuthology.orchestra.run.smithi145.stdout: "roption2": { 2023-08-25T05:00:47.884 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption2", 2023-08-25T05:00:47.885 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.885 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.885 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.885 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "xyz", 2023-08-25T05:00:47.885 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.886 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.886 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.886 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.886 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.887 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.887 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.888 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption1": { 2023-08-25T05:00:47.888 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption1", 2023-08-25T05:00:47.888 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.889 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.889 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.889 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.889 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.889 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.890 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.890 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.890 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.890 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.890 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.891 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption2": { 2023-08-25T05:00:47.891 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption2", 2023-08-25T05:00:47.891 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.892 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.892 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.892 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.892 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.893 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.893 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.893 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.894 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.894 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.894 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.894 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.894 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption3": { 2023-08-25T05:00:47.895 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption3", 2023-08-25T05:00:47.895 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:47.895 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.895 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.895 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.896 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.897 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.897 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.897 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption4": { 2023-08-25T05:00:47.897 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption4", 2023-08-25T05:00:47.897 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.898 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.898 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.898 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.898 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.898 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.899 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.899 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.899 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.899 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.899 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.900 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption5": { 2023-08-25T05:00:47.900 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption5", 2023-08-25T05:00:47.900 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.901 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.901 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.901 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.901 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.902 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.903 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.903 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption6": { 2023-08-25T05:00:47.903 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption6", 2023-08-25T05:00:47.903 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.903 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.904 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.904 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.905 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.905 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.905 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.905 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.905 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption7": { 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption7", 2023-08-25T05:00:47.906 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:47.907 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.907 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.907 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.907 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:47.908 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "42", 2023-08-25T05:00:47.908 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.908 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.908 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.908 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.909 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.909 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.909 INFO:teuthology.orchestra.run.smithi145.stdout: "testkey": { 2023-08-25T05:00:47.909 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testkey", 2023-08-25T05:00:47.909 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.910 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.910 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.910 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.910 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.910 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.911 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.911 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.911 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.911 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.911 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.912 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.912 INFO:teuthology.orchestra.run.smithi145.stdout: "testlkey": { 2023-08-25T05:00:47.912 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testlkey", 2023-08-25T05:00:47.913 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.913 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.913 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.913 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.913 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.914 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.915 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.915 INFO:teuthology.orchestra.run.smithi145.stdout: "testnewline": { 2023-08-25T05:00:47.915 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testnewline", 2023-08-25T05:00:47.915 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.916 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.916 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.916 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.916 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.917 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.917 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.917 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.917 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.917 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.918 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.918 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.918 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.918 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.918 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snap_schedule", 2023-08-25T05:00:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.919 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_m_granularity": { 2023-08-25T05:00:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_m_granularity", 2023-08-25T05:00:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.920 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T05:00:47.921 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "dump_on_update": { 2023-08-25T05:00:47.922 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dump_on_update", 2023-08-25T05:00:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.923 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "dump database to debug log on update", 2023-08-25T05:00:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.924 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.925 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.926 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.927 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.928 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.929 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.930 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.931 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.932 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.933 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.934 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.935 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.936 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.937 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.938 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.939 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.940 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stats", 2023-08-25T05:00:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.941 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.942 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.943 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.944 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.945 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.946 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.947 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.948 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.949 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.950 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.951 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.952 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.953 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.954 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.955 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.956 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.957 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "status", 2023-08-25T05:00:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.958 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.959 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.960 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.961 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.962 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.964 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.965 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.966 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.967 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.968 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.969 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.970 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.971 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.972 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.973 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telegraf", 2023-08-25T05:00:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "address": { 2023-08-25T05:00:47.974 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "address", 2023-08-25T05:00:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.975 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.976 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:47.977 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15", 2023-08-25T05:00:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.978 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.979 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.980 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.981 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.982 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.983 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:47.984 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.985 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.986 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.987 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.988 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:47.989 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.990 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:47.991 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:47.992 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.993 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.994 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telemetry", 2023-08-25T05:00:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:47.995 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_basic": { 2023-08-25T05:00:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_basic", 2023-08-25T05:00:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.996 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T05:00:47.997 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_crash": { 2023-08-25T05:00:47.998 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_crash", 2023-08-25T05:00:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:47.999 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.000 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.000 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.000 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T05:00:48.000 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.001 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.001 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.002 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_device": { 2023-08-25T05:00:48.002 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_device", 2023-08-25T05:00:48.002 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.002 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.002 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.003 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.003 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.003 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.003 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.003 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_ident": { 2023-08-25T05:00:48.004 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_ident", 2023-08-25T05:00:48.005 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.005 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.005 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.005 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.005 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.006 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.006 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.006 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T05:00:48.006 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.006 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.007 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.007 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.007 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_perf": { 2023-08-25T05:00:48.007 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_perf", 2023-08-25T05:00:48.007 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.008 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.008 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.008 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.008 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.008 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.009 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.009 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T05:00:48.009 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.009 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.009 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.010 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.010 INFO:teuthology.orchestra.run.smithi145.stdout: "contact": { 2023-08-25T05:00:48.010 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "contact", 2023-08-25T05:00:48.010 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.010 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.011 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.011 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.011 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.011 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.011 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.012 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.012 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.012 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.012 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.012 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.013 INFO:teuthology.orchestra.run.smithi145.stdout: "description": { 2023-08-25T05:00:48.013 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "description", 2023-08-25T05:00:48.013 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.013 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.013 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.014 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.014 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.014 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.014 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.014 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.015 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.015 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.015 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.015 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.015 INFO:teuthology.orchestra.run.smithi145.stdout: "device_url": { 2023-08-25T05:00:48.016 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_url", 2023-08-25T05:00:48.016 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.016 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.016 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.016 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T05:00:48.017 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.017 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.017 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.017 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.017 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.018 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.018 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.018 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:48.018 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.020 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.020 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.020 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.020 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.021 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.021 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.021 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:48.022 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:48.022 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.022 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.022 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.023 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "24", 2023-08-25T05:00:48.023 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "8", 2023-08-25T05:00:48.023 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.023 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.023 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.024 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.024 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.024 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.024 INFO:teuthology.orchestra.run.smithi145.stdout: "last_opt_revision": { 2023-08-25T05:00:48.025 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "last_opt_revision", 2023-08-25T05:00:48.025 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.025 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.025 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.025 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1", 2023-08-25T05:00:48.026 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.026 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.026 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.026 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.026 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.027 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.027 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.027 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.027 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard": { 2023-08-25T05:00:48.027 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard", 2023-08-25T05:00:48.028 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.028 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.028 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.028 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.029 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.029 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.029 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.029 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.029 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.030 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.030 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.030 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.030 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard_description": { 2023-08-25T05:00:48.030 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard_description", 2023-08-25T05:00:48.031 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.031 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.031 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.031 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.031 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.032 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.032 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.032 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.032 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.032 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.033 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.033 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.033 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.033 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.034 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.034 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.034 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.034 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.034 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.035 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.036 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.036 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.036 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.036 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.036 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.037 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.037 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.037 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.037 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.037 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.038 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.038 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.038 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.038 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.038 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.039 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.039 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.039 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.039 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.039 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.040 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.040 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.040 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.040 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.041 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.041 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.041 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.041 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.041 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.042 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.042 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.042 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.042 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.042 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.043 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.043 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.043 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.043 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.043 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.044 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.044 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.044 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.044 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.045 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.045 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.045 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.045 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.045 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.046 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.047 INFO:teuthology.orchestra.run.smithi145.stdout: "organization": { 2023-08-25T05:00:48.047 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "organization", 2023-08-25T05:00:48.047 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.047 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.048 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.048 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.048 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.048 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.049 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.049 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.049 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.049 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.049 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.050 INFO:teuthology.orchestra.run.smithi145.stdout: "proxy": { 2023-08-25T05:00:48.050 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "proxy", 2023-08-25T05:00:48.050 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.050 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.051 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.051 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.051 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.051 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.051 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.052 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.052 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.052 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.052 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.053 INFO:teuthology.orchestra.run.smithi145.stdout: "url": { 2023-08-25T05:00:48.053 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url", 2023-08-25T05:00:48.053 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.053 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.053 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.054 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T05:00:48.054 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.054 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.054 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.054 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.055 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.055 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.055 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.056 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.056 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.057 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "test_orchestrator", 2023-08-25T05:00:48.057 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.057 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.057 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.058 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.058 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.058 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.059 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.059 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.059 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.060 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.060 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.060 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.060 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.061 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.062 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.062 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.062 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.062 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.062 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.063 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.063 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.063 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.063 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.064 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.064 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.064 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.064 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.064 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.065 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.065 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.065 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.065 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.066 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.066 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.066 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.066 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.067 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.067 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.067 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.067 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.067 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.068 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.068 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.068 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.068 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.068 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.069 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.069 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.069 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.069 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.070 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.070 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.070 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.070 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.072 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.072 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.072 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.072 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.072 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.073 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.073 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.073 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.074 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "volumes", 2023-08-25T05:00:48.074 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.074 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.074 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.074 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.075 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.075 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.075 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.075 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.075 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.076 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.076 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.076 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.076 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.076 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.077 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.077 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.077 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.077 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.077 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.078 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.078 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.079 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.079 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.079 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.080 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.080 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.080 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.080 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.081 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.081 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.081 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.081 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.081 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.082 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.082 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.082 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.082 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.083 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.083 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.083 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.083 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.083 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.084 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.084 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.084 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.084 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.084 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.085 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.085 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.085 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.085 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.085 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.086 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.086 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.086 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.086 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.086 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.087 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.087 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.087 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.087 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.087 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.088 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.088 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.088 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.088 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.088 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_clones": { 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_clones", 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.090 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4", 2023-08-25T05:00:48.091 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.091 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.091 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.091 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T05:00:48.092 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.092 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.092 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.092 INFO:teuthology.orchestra.run.smithi145.stdout: "snapshot_clone_delay": { 2023-08-25T05:00:48.093 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snapshot_clone_delay", 2023-08-25T05:00:48.093 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.093 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.093 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.093 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.094 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.095 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.095 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.095 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.095 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.095 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.096 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix", 2023-08-25T05:00:48.096 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.096 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.096 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.096 INFO:teuthology.orchestra.run.smithi145.stdout: "discovery_interval": { 2023-08-25T05:00:48.097 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "discovery_interval", 2023-08-25T05:00:48.097 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:48.097 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.097 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.097 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "100", 2023-08-25T05:00:48.098 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.098 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.098 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.098 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.098 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.099 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.099 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.099 INFO:teuthology.orchestra.run.smithi145.stdout: "identifier": { 2023-08-25T05:00:48.099 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "identifier", 2023-08-25T05:00:48.100 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.100 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.100 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.100 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.100 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.101 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.101 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.101 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.102 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.102 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.102 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.102 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.102 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:48.103 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.104 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.104 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.104 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.104 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.105 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.106 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.106 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.106 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.106 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.106 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.107 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.107 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.107 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.107 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.107 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.108 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.108 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.108 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.108 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.108 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.109 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.109 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.109 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.109 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.109 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.110 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.110 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.110 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.110 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.110 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.111 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.111 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.111 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.111 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.111 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.112 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.600 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.600 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.600 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.601 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.601 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.601 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.601 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.602 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.602 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.602 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.602 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.603 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.603 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.604 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.604 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.604 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.604 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.605 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.605 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.605 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.605 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.606 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.606 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.606 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.606 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.607 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.607 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.607 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.608 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.608 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.608 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.608 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.609 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.609 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.609 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.609 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_host": { 2023-08-25T05:00:48.610 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_host", 2023-08-25T05:00:48.610 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.610 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.610 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.611 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.611 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.611 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.611 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.612 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.612 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.612 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.612 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.613 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.614 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_port": { 2023-08-25T05:00:48.614 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_port", 2023-08-25T05:00:48.614 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.615 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.615 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.615 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10051", 2023-08-25T05:00:48.615 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.615 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.616 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.616 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.616 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.616 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.617 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.618 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.618 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_sender": { 2023-08-25T05:00:48.618 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_sender", 2023-08-25T05:00:48.618 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.619 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.619 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.619 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T05:00:48.619 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.620 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.621 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.621 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.621 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.622 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.622 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.622 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.622 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.623 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.623 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:48.623 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.623 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.623 INFO:teuthology.orchestra.run.smithi145.stdout: "gid": 25501, 2023-08-25T05:00:48.624 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "z", 2023-08-25T05:00:48.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_features": 4540138322906710015, 2023-08-25T05:00:48.624 INFO:teuthology.orchestra.run.smithi145.stdout: "available_modules": [ 2023-08-25T05:00:48.625 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.625 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "alerts", 2023-08-25T05:00:48.625 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.625 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.626 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.626 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:48.626 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:48.626 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.626 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.627 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.627 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:48.627 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.627 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.628 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.628 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T05:00:48.628 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.629 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.629 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.629 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.629 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.630 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.630 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.630 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.630 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.631 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.631 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.631 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.631 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.632 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.632 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.632 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.633 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.633 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.633 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.633 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.633 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.634 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.634 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.634 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.634 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.635 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.635 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.636 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.636 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.637 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.637 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.637 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.638 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.638 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.638 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.638 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.638 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.639 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.639 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.639 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.639 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.640 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.640 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.640 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.640 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.641 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.641 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.641 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.642 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.642 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.642 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.643 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.643 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.643 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.643 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.644 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.644 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.644 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.644 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.645 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.645 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.645 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.646 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.646 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.646 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.646 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.646 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.647 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.647 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.648 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.648 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.648 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.648 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.649 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_destination": { 2023-08-25T05:00:48.649 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_destination", 2023-08-25T05:00:48.649 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.649 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.650 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.650 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.650 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.651 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.651 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.651 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email address to send alerts to", 2023-08-25T05:00:48.651 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.652 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.652 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.652 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.652 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_from_name": { 2023-08-25T05:00:48.653 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_from_name", 2023-08-25T05:00:48.653 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.653 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.653 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.653 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "Ceph", 2023-08-25T05:00:48.654 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.654 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.654 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.654 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email From: name", 2023-08-25T05:00:48.654 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.655 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.655 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.655 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.655 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_host": { 2023-08-25T05:00:48.655 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_host", 2023-08-25T05:00:48.656 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.656 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.656 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.656 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.656 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.657 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.657 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.657 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP server", 2023-08-25T05:00:48.657 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.658 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.658 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.658 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.658 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_password": { 2023-08-25T05:00:48.658 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_password", 2023-08-25T05:00:48.659 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.659 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.659 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.659 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.659 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.660 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.660 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.660 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Password to authenticate with", 2023-08-25T05:00:48.660 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.661 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.661 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.661 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_port": { 2023-08-25T05:00:48.661 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_port", 2023-08-25T05:00:48.662 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.662 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.662 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.662 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "465", 2023-08-25T05:00:48.662 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.663 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.663 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.663 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP port", 2023-08-25T05:00:48.663 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.663 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.664 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.664 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_sender": { 2023-08-25T05:00:48.664 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_sender", 2023-08-25T05:00:48.664 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.665 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.665 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.665 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.665 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.665 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.666 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.666 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP envelope sender", 2023-08-25T05:00:48.666 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.666 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.666 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.667 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_ssl": { 2023-08-25T05:00:48.667 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_ssl", 2023-08-25T05:00:48.667 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.667 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.668 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.668 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.668 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.668 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.668 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.669 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T05:00:48.669 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.669 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.669 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.670 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_user": { 2023-08-25T05:00:48.670 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_user", 2023-08-25T05:00:48.670 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.670 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.671 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.671 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.671 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.671 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.672 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.672 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "User to authenticate as", 2023-08-25T05:00:48.672 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.672 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.673 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "balancer", 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.674 INFO:teuthology.orchestra.run.smithi145.stdout: "active": { 2023-08-25T05:00:48.675 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "active", 2023-08-25T05:00:48.675 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.675 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.675 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.676 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.676 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.676 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.676 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.677 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T05:00:48.677 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.677 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.677 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.678 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_time": { 2023-08-25T05:00:48.678 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_time", 2023-08-25T05:00:48.678 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.679 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.679 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.679 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0000", 2023-08-25T05:00:48.679 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.679 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.680 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.680 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T05:00:48.680 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:48.681 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.681 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.681 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_weekday": { 2023-08-25T05:00:48.682 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_weekday", 2023-08-25T05:00:48.682 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:48.682 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.682 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.683 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:48.683 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:48.683 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:48.683 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.684 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T05:00:48.684 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:48.684 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.684 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.685 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_max_iterations": { 2023-08-25T05:00:48.685 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_max_iterations", 2023-08-25T05:00:48.685 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:48.686 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.686 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.686 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "25", 2023-08-25T05:00:48.686 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "250", 2023-08-25T05:00:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.687 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T05:00:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_metrics": { 2023-08-25T05:00:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_metrics", 2023-08-25T05:00:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.689 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.690 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.690 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T05:00:48.690 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.690 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.691 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.691 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T05:00:48.691 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T05:00:48.692 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.692 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.692 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.692 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_step": { 2023-08-25T05:00:48.693 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_step", 2023-08-25T05:00:48.693 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:48.693 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.693 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.694 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.5", 2023-08-25T05:00:48.694 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0.001", 2023-08-25T05:00:48.695 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "0.999", 2023-08-25T05:00:48.695 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.695 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "aggressiveness of optimization", 2023-08-25T05:00:48.695 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T05:00:48.696 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.696 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.696 INFO:teuthology.orchestra.run.smithi145.stdout: "end_time": { 2023-08-25T05:00:48.697 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_time", 2023-08-25T05:00:48.697 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.697 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.697 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.698 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2359", 2023-08-25T05:00:48.698 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.698 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.698 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.699 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T05:00:48.699 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:48.699 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.699 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.700 INFO:teuthology.orchestra.run.smithi145.stdout: "end_weekday": { 2023-08-25T05:00:48.700 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_weekday", 2023-08-25T05:00:48.701 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:48.701 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.701 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.701 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:48.702 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:48.702 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:48.702 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.705 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T05:00:48.705 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:48.705 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.706 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.706 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.706 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.707 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.707 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.707 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.707 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.708 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.708 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.708 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.708 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.709 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.709 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.709 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.709 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.710 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.710 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.710 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.710 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.710 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.711 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.711 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.711 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.712 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.712 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.712 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.712 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.713 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.713 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.713 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.713 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.714 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.714 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.714 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.714 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.715 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.715 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.715 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.715 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.716 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.716 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.716 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.716 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.717 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.717 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.717 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.717 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.718 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.718 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.718 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.718 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.718 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.719 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.719 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.719 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.719 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.719 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.720 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.720 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.720 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.720 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.720 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.721 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.721 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.721 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.721 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.722 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.722 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.722 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.722 INFO:teuthology.orchestra.run.smithi145.stdout: "min_score": { 2023-08-25T05:00:48.722 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_score", 2023-08-25T05:00:48.723 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:48.723 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.723 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.724 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:48.724 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.724 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.724 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.725 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T05:00:48.725 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.725 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.725 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.726 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:48.726 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:48.726 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.727 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.727 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.727 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "upmap", 2023-08-25T05:00:48.728 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.728 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.728 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.728 INFO:teuthology.orchestra.run.smithi145.stdout: "crush-compat", 2023-08-25T05:00:48.729 INFO:teuthology.orchestra.run.smithi145.stdout: "none", 2023-08-25T05:00:48.729 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap" 2023-08-25T05:00:48.729 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.729 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Balancer mode", 2023-08-25T05:00:48.730 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.730 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.730 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.734 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ids": { 2023-08-25T05:00:48.734 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_ids", 2023-08-25T05:00:48.734 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.734 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.734 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.735 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.735 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.735 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.735 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.735 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T05:00:48.736 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.737 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.737 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.738 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:48.738 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:48.738 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.739 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.740 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.740 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:48.740 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.741 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.741 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.741 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T05:00:48.741 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.742 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.742 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.743 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_deviation": { 2023-08-25T05:00:48.744 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_deviation", 2023-08-25T05:00:48.745 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.745 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.745 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.745 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:48.746 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:48.746 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.746 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.748 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T05:00:48.748 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T05:00:48.749 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.751 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.752 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.753 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_optimizations": { 2023-08-25T05:00:48.753 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_optimizations", 2023-08-25T05:00:48.754 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:48.755 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.756 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:48.757 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.757 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.757 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.757 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T05:00:48.758 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.758 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.758 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.758 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.758 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.759 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.759 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.759 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cephadm", 2023-08-25T05:00:48.759 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.760 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.760 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.761 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_down_multiplier": { 2023-08-25T05:00:48.761 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_down_multiplier", 2023-08-25T05:00:48.761 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:48.761 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.762 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.762 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:48.762 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.762 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.763 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.763 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T05:00:48.763 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.763 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.763 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.764 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_refresh_rate": { 2023-08-25T05:00:48.764 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_refresh_rate", 2023-08-25T05:00:48.764 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.765 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.765 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.765 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "20", 2023-08-25T05:00:48.765 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.765 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.766 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.766 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T05:00:48.766 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.766 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.766 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.767 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_starting_port": { 2023-08-25T05:00:48.767 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_starting_port", 2023-08-25T05:00:48.767 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.767 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.768 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.768 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4721", 2023-08-25T05:00:48.768 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.768 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.768 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.769 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T05:00:48.769 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.769 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.769 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.769 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.770 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_ptrace": { 2023-08-25T05:00:48.770 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_ptrace", 2023-08-25T05:00:48.770 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.770 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.770 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T05:00:48.771 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T05:00:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_interval": { 2023-08-25T05:00:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_interval", 2023-08-25T05:00:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T05:00:48.774 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_memory_target_ratio": { 2023-08-25T05:00:48.775 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T05:00:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.7", 2023-08-25T05:00:48.776 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T05:00:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.777 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: "cgroups_split": { 2023-08-25T05:00:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cgroups_split", 2023-08-25T05:00:48.778 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.779 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T05:00:48.780 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.781 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.781 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.781 INFO:teuthology.orchestra.run.smithi145.stdout: "config_checks_enabled": { 2023-08-25T05:00:48.781 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_checks_enabled", 2023-08-25T05:00:48.782 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.782 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.782 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.782 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.782 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.783 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.783 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.783 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T05:00:48.783 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.783 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.784 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.784 INFO:teuthology.orchestra.run.smithi145.stdout: "config_dashboard": { 2023-08-25T05:00:48.784 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_dashboard", 2023-08-25T05:00:48.784 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.785 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.785 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.785 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.785 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.785 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.786 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.786 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T05:00:48.786 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.786 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.786 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.787 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_alertmanager": { 2023-08-25T05:00:48.787 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_alertmanager", 2023-08-25T05:00:48.787 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.787 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.788 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.788 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T05:00:48.788 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.788 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.788 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.789 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:48.789 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.789 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.789 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.790 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_base": { 2023-08-25T05:00:48.790 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_base", 2023-08-25T05:00:48.790 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.790 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.791 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.791 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T05:00:48.791 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.791 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.791 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.792 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Container image name, without the tag", 2023-08-25T05:00:48.792 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.792 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.792 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.793 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_elasticsearch": { 2023-08-25T05:00:48.793 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_elasticsearch", 2023-08-25T05:00:48.793 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.793 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.793 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.794 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T05:00:48.794 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.794 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.794 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.794 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "elasticsearch container image", 2023-08-25T05:00:48.795 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.795 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.795 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.795 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_grafana": { 2023-08-25T05:00:48.796 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_grafana", 2023-08-25T05:00:48.796 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.796 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.796 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.796 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T05:00:48.797 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.797 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.797 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.797 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:48.797 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.798 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.798 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.798 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_haproxy": { 2023-08-25T05:00:48.799 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_haproxy", 2023-08-25T05:00:48.799 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.799 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.799 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.800 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T05:00:48.800 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.800 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.801 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.801 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "HAproxy container image", 2023-08-25T05:00:48.801 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.801 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_agent": { 2023-08-25T05:00:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_agent", 2023-08-25T05:00:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T05:00:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger agent container image", 2023-08-25T05:00:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.805 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.805 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_collector": { 2023-08-25T05:00:48.805 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_collector", 2023-08-25T05:00:48.805 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.806 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.806 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.806 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T05:00:48.806 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.806 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.807 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.807 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger collector container image", 2023-08-25T05:00:48.807 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.807 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.807 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.808 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.808 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_query": { 2023-08-25T05:00:48.808 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_query", 2023-08-25T05:00:48.808 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.809 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.809 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.809 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T05:00:48.809 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.809 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.810 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.811 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger query container image", 2023-08-25T05:00:48.811 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.811 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.811 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.811 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.812 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_keepalived": { 2023-08-25T05:00:48.812 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_keepalived", 2023-08-25T05:00:48.812 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.812 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.812 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.813 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T05:00:48.813 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.813 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.813 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.813 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Keepalived container image", 2023-08-25T05:00:48.814 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.814 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.814 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.814 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_loki": { 2023-08-25T05:00:48.815 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_loki", 2023-08-25T05:00:48.815 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.815 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.815 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.815 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T05:00:48.816 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.816 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.821 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.821 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Loki container image", 2023-08-25T05:00:48.821 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.821 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.822 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.822 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_node_exporter": { 2023-08-25T05:00:48.822 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_node_exporter", 2023-08-25T05:00:48.822 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.823 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.823 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.823 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T05:00:48.824 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.824 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.824 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.825 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:48.825 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.825 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.825 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.826 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_nvmeof": { 2023-08-25T05:00:48.826 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_nvmeof", 2023-08-25T05:00:48.826 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.827 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.827 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.827 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T05:00:48.828 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.828 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.829 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.829 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Nvme-of container image", 2023-08-25T05:00:48.829 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.829 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.830 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.830 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_prometheus": { 2023-08-25T05:00:48.830 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_prometheus", 2023-08-25T05:00:48.831 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.831 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.831 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.831 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T05:00:48.831 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.832 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.832 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.832 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:48.833 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.833 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.833 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.833 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.834 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_promtail": { 2023-08-25T05:00:48.834 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_promtail", 2023-08-25T05:00:48.834 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.834 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.835 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.835 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T05:00:48.835 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.835 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.836 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.836 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Promtail container image", 2023-08-25T05:00:48.837 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.837 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.837 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.837 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.837 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_snmp_gateway": { 2023-08-25T05:00:48.838 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_snmp_gateway", 2023-08-25T05:00:48.838 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.838 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.838 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.839 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T05:00:48.839 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.839 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.839 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.839 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SNMP Gateway container image", 2023-08-25T05:00:48.840 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.840 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.840 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.840 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.841 INFO:teuthology.orchestra.run.smithi145.stdout: "container_init": { 2023-08-25T05:00:48.841 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_init", 2023-08-25T05:00:48.841 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.841 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.842 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.842 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.842 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.842 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.842 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.843 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T05:00:48.843 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.844 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.844 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.845 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.845 INFO:teuthology.orchestra.run.smithi145.stdout: "daemon_cache_timeout": { 2023-08-25T05:00:48.845 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "daemon_cache_timeout", 2023-08-25T05:00:48.846 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.846 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.846 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.847 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:48.847 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.847 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.847 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.848 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T05:00:48.848 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.848 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.848 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.848 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.849 INFO:teuthology.orchestra.run.smithi145.stdout: "default_cephadm_command_timeout": { 2023-08-25T05:00:48.849 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T05:00:48.849 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.849 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.849 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.850 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "900", 2023-08-25T05:00:48.850 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.850 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.850 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.851 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T05:00:48.851 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.851 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.851 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.852 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.852 INFO:teuthology.orchestra.run.smithi145.stdout: "default_registry": { 2023-08-25T05:00:48.852 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_registry", 2023-08-25T05:00:48.854 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.855 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.855 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.855 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io", 2023-08-25T05:00:48.856 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.856 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.857 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.857 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T05:00:48.858 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.858 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.858 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.859 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.859 INFO:teuthology.orchestra.run.smithi145.stdout: "device_cache_timeout": { 2023-08-25T05:00:48.860 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_cache_timeout", 2023-08-25T05:00:48.860 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.861 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.861 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.861 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1800", 2023-08-25T05:00:48.862 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.862 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.862 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.863 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache device inventory", 2023-08-25T05:00:48.863 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.863 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.865 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.865 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.865 INFO:teuthology.orchestra.run.smithi145.stdout: "device_enhanced_scan": { 2023-08-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_enhanced_scan", 2023-08-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.868 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.868 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T05:00:48.869 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.869 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.869 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.870 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.870 INFO:teuthology.orchestra.run.smithi145.stdout: "facts_cache_timeout": { 2023-08-25T05:00:48.870 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "facts_cache_timeout", 2023-08-25T05:00:48.871 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.871 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.871 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.872 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:48.872 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.873 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.873 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.873 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache host facts data", 2023-08-25T05:00:48.874 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.874 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.874 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.875 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.876 INFO:teuthology.orchestra.run.smithi145.stdout: "host_check_interval": { 2023-08-25T05:00:48.876 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "host_check_interval", 2023-08-25T05:00:48.876 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:48.876 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.877 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.877 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:48.878 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.878 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.878 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.878 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to perform a host check", 2023-08-25T05:00:48.879 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.879 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.879 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.882 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.882 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.882 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.882 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.883 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.883 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.883 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.883 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.884 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.884 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.884 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.885 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.885 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.885 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.885 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.886 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.886 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.886 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.886 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.887 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.887 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.887 INFO:teuthology.orchestra.run.smithi145.stdout: "log_refresh_metadata": { 2023-08-25T05:00:48.887 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_refresh_metadata", 2023-08-25T05:00:48.888 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.888 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.888 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.888 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.889 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.889 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.889 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.889 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T05:00:48.890 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.890 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.890 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.891 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:48.891 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:48.891 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.891 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.892 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.892 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.892 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.892 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.892 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.893 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T05:00:48.893 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.893 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.893 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.894 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.894 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:48.894 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:48.894 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.895 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.895 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.895 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:48.895 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.896 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.896 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.896 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:48.896 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:48.896 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:48.897 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:48.897 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:48.897 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:48.897 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.897 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.898 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.898 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.898 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.899 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:48.899 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:48.899 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.899 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.899 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.900 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.900 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.900 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.900 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.900 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:48.901 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.901 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.901 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.902 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T05:00:48.902 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T05:00:48.902 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.902 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.902 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.903 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.903 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.903 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.903 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.903 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T05:00:48.904 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.904 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.904 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.904 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.905 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T05:00:48.905 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T05:00:48.905 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.905 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.906 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.906 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "*", 2023-08-25T05:00:48.906 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.906 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.907 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.907 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T05:00:48.907 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.907 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.908 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.908 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.912 INFO:teuthology.orchestra.run.smithi145.stdout: "max_count_per_host": { 2023-08-25T05:00:48.913 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_count_per_host", 2023-08-25T05:00:48.913 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.913 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.913 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.914 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:48.914 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.914 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.915 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.915 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of daemons per service per host", 2023-08-25T05:00:48.915 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.915 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.916 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.916 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.916 INFO:teuthology.orchestra.run.smithi145.stdout: "max_osd_draining_count": { 2023-08-25T05:00:48.917 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_osd_draining_count", 2023-08-25T05:00:48.917 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.917 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.917 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.918 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:48.918 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.918 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.918 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.919 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T05:00:48.919 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.919 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.920 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.920 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.920 INFO:teuthology.orchestra.run.smithi145.stdout: "migration_current": { 2023-08-25T05:00:48.921 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "migration_current", 2023-08-25T05:00:48.921 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.921 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.921 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.922 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.922 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.922 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.923 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.923 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "internal - do not modify", 2023-08-25T05:00:48.923 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.923 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.928 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.928 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.929 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:48.929 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:48.929 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.929 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.929 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.930 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "root", 2023-08-25T05:00:48.930 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.930 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.930 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:48.930 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm-package", 2023-08-25T05:00:48.931 INFO:teuthology.orchestra.run.smithi145.stdout: "root" 2023-08-25T05:00:48.931 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:48.931 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T05:00:48.931 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.931 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.932 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.932 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.932 INFO:teuthology.orchestra.run.smithi145.stdout: "prometheus_alerts_path": { 2023-08-25T05:00:48.932 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus_alerts_path", 2023-08-25T05:00:48.938 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.938 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.938 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.938 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T05:00:48.939 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.939 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.939 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.939 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T05:00:48.940 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.940 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.946 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.946 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.946 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_insecure": { 2023-08-25T05:00:48.947 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_insecure", 2023-08-25T05:00:48.947 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.947 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.948 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.948 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.948 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.949 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.949 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.949 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T05:00:48.949 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.949 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.950 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.950 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.950 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_password": { 2023-08-25T05:00:48.950 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_password", 2023-08-25T05:00:48.950 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.951 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.951 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.951 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.951 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.951 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.952 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.952 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T05:00:48.952 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.952 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.952 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.953 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.953 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_url": { 2023-08-25T05:00:48.953 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_url", 2023-08-25T05:00:48.953 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.953 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.954 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.954 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.954 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.954 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.954 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.955 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T05:00:48.955 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.955 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.955 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.956 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_username": { 2023-08-25T05:00:48.956 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_username", 2023-08-25T05:00:48.956 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.956 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.956 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.957 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.957 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.957 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.957 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.958 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T05:00:48.958 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.958 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.959 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.959 INFO:teuthology.orchestra.run.smithi145.stdout: "secure_monitoring_stack": { 2023-08-25T05:00:48.959 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "secure_monitoring_stack", 2023-08-25T05:00:48.959 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.960 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.960 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.960 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.960 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.960 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.961 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.961 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T05:00:48.961 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.961 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.961 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.962 INFO:teuthology.orchestra.run.smithi145.stdout: "service_discovery_port": { 2023-08-25T05:00:48.962 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "service_discovery_port", 2023-08-25T05:00:48.962 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:48.962 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.963 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.963 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8765", 2023-08-25T05:00:48.963 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.963 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.963 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.964 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "cephadm service discovery port", 2023-08-25T05:00:48.964 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.964 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.964 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.965 INFO:teuthology.orchestra.run.smithi145.stdout: "ssh_config_file": { 2023-08-25T05:00:48.965 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssh_config_file", 2023-08-25T05:00:48.965 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.965 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.966 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.966 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.966 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.966 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.967 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.968 INFO:teuthology.orchestra.run.smithi145.stdout: "use_agent": { 2023-08-25T05:00:48.968 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_agent", 2023-08-25T05:00:48.969 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.969 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.969 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.969 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:48.970 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.970 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.975 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.976 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T05:00:48.976 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.976 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.977 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.977 INFO:teuthology.orchestra.run.smithi145.stdout: "use_repo_digest": { 2023-08-25T05:00:48.977 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_repo_digest", 2023-08-25T05:00:48.977 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.978 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.978 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.978 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.978 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.978 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T05:00:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.980 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_failed_host_check": { 2023-08-25T05:00:48.985 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_failed_host_check", 2023-08-25T05:00:48.986 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.986 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.986 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.986 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.987 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.987 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.987 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.987 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T05:00:48.987 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.988 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.988 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.988 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_daemons": { 2023-08-25T05:00:48.988 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_daemons", 2023-08-25T05:00:48.989 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.989 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.989 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.989 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.989 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.990 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.990 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.990 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T05:00:48.990 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.991 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.991 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.991 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_hosts": { 2023-08-25T05:00:48.992 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_hosts", 2023-08-25T05:00:48.992 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:48.992 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.992 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T05:00:48.993 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:48.994 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:48.994 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:48.994 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.994 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:48.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:48.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:48.997 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crash", 2023-08-25T05:00:48.997 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:48.997 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:48.999 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:48.999 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:48.999 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:48.999 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.000 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.000 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.001 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.001 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.001 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.002 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.002 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.002 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.003 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.003 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.003 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.003 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.004 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.004 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.004 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.005 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.005 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.005 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.005 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.006 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.006 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.006 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.006 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.007 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.007 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.008 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.008 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.008 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.008 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.009 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.009 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.012 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.012 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.013 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.013 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.013 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.014 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.014 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.014 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.015 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.015 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.015 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.015 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.016 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.016 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.016 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.016 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.017 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.017 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.017 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.017 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.017 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.018 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.018 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.019 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.019 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.019 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.019 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.019 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.020 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.020 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.020 INFO:teuthology.orchestra.run.smithi145.stdout: "retain_interval": { 2023-08-25T05:00:49.020 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retain_interval", 2023-08-25T05:00:49.021 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.021 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.021 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.022 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "31536000", 2023-08-25T05:00:49.022 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.022 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.023 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.024 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_recent_interval": { 2023-08-25T05:00:49.024 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_recent_interval", 2023-08-25T05:00:49.024 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.025 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.025 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.025 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1209600", 2023-08-25T05:00:49.025 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.025 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.026 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.026 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T05:00:49.026 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.026 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.027 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dashboard", 2023-08-25T05:00:49.028 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.028 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.028 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.028 INFO:teuthology.orchestra.run.smithi145.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.030 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.030 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.030 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.030 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.030 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.031 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.031 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T05:00:49.031 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T05:00:49.031 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.032 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.032 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.033 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.033 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.033 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.034 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.034 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.034 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.035 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.035 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.035 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T05:00:49.035 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T05:00:49.036 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.036 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.036 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.036 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.037 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.037 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.037 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.038 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.038 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.038 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.038 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.039 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_ENABLED": { 2023-08-25T05:00:49.039 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T05:00:49.039 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.039 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.040 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.040 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.040 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.040 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.041 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.041 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.041 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.041 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.041 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.042 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T05:00:49.042 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T05:00:49.042 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.043 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.043 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.043 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.043 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.044 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.044 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.044 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.045 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.045 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.045 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.046 INFO:teuthology.orchestra.run.smithi145.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T05:00:49.046 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T05:00:49.046 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.046 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.046 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.047 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.047 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.047 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.047 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.048 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.048 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.048 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.048 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.049 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T05:00:49.049 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T05:00:49.049 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.049 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.050 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.050 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.050 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.050 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.050 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.051 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.051 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.051 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.051 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.052 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T05:00:49.052 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T05:00:49.052 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.053 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.053 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.054 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.054 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.054 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.054 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.054 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.055 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.055 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.055 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.056 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T05:00:49.056 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T05:00:49.057 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.057 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.057 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.058 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.058 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.058 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.058 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.058 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.059 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.059 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.059 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.060 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T05:00:49.060 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T05:00:49.060 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.060 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.061 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.061 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.061 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.061 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.062 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.062 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.062 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.062 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.062 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.063 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T05:00:49.063 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T05:00:49.063 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.063 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.064 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.064 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.064 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.064 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.065 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.065 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.065 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.065 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.065 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.066 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T05:00:49.066 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T05:00:49.066 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.066 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.067 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.067 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.067 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.068 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.068 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.068 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.069 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.069 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.069 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.070 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T05:00:49.070 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T05:00:49.070 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.070 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.070 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.071 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.072 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.072 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.072 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.072 INFO:teuthology.orchestra.run.smithi145.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T05:00:49.072 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T05:00:49.073 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.073 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.073 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.074 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.074 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.074 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.075 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.075 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.075 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.075 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.075 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.078 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T05:00:49.079 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T05:00:49.079 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.079 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.079 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.080 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:49.080 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.080 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.080 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.081 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.081 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.081 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.081 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.086 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.086 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T05:00:49.086 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T05:00:49.087 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.087 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.087 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.087 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.087 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.088 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.088 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.088 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.088 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.089 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.089 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.089 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_URL": { 2023-08-25T05:00:49.089 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_URL", 2023-08-25T05:00:49.090 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.090 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.090 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.090 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.090 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.091 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.091 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.091 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.091 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.091 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.092 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.092 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T05:00:49.092 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T05:00:49.092 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.093 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.094 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.094 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.094 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.094 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.095 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T05:00:49.095 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T05:00:49.095 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.095 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.095 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.096 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.096 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.097 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.097 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.100 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.100 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.101 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.101 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.101 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T05:00:49.102 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T05:00:49.102 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.102 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.102 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.103 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.103 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.103 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.103 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.103 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.104 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.104 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.104 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.104 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.104 INFO:teuthology.orchestra.run.smithi145.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T05:00:49.105 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T05:00:49.105 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.105 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.105 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.105 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.106 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.106 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.106 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.106 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.107 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.107 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.107 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.108 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.108 INFO:teuthology.orchestra.run.smithi145.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T05:00:49.108 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T05:00:49.108 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.109 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.109 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.109 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.109 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.109 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.110 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.110 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.110 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.110 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.110 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.111 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.111 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T05:00:49.111 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T05:00:49.111 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.111 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.112 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.112 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.112 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.112 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.114 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T05:00:49.114 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T05:00:49.114 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.114 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.115 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.115 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.115 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.115 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.115 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.116 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.116 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.116 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.116 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.116 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.117 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T05:00:49.117 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T05:00:49.117 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.118 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.118 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.118 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.118 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.119 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.119 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.119 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.119 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.119 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.120 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.120 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.120 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T05:00:49.120 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T05:00:49.121 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.121 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.121 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.121 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.122 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.122 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.122 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.122 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.122 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.123 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.123 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.123 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.123 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T05:00:49.123 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T05:00:49.124 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.124 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.124 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.124 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.124 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.125 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.125 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.125 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.125 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.125 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.126 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.126 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T05:00:49.126 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T05:00:49.126 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.127 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.127 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.127 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.127 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.127 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.128 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.128 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.128 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.129 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.129 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.129 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T05:00:49.129 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T05:00:49.130 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.130 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.130 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.130 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.130 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.131 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.131 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.131 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.131 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.132 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.134 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.134 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.135 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T05:00:49.135 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T05:00:49.135 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.136 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.136 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.137 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.140 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.140 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.141 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T05:00:49.141 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T05:00:49.141 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.141 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.142 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.142 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.142 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.142 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.142 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.145 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.145 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.146 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.146 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.146 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T05:00:49.147 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T05:00:49.147 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.147 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.147 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.148 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.148 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.148 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.148 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.148 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.149 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.149 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.150 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.150 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T05:00:49.151 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T05:00:49.151 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.151 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.151 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.151 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T05:00:49.152 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.152 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.152 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.152 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.152 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.153 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.153 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.153 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T05:00:49.154 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T05:00:49.154 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.154 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.155 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.155 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:49.155 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.156 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.156 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.156 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.156 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.157 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.157 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.157 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T05:00:49.157 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T05:00:49.158 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.158 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.158 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.158 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8", 2023-08-25T05:00:49.158 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.159 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.159 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.159 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.159 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.159 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.160 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.160 INFO:teuthology.orchestra.run.smithi145.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T05:00:49.160 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T05:00:49.160 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.161 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.161 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.161 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "45", 2023-08-25T05:00:49.161 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.161 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T05:00:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T05:00:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.164 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.164 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.164 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.164 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.164 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.165 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.165 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.165 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.166 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.166 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T05:00:49.167 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T05:00:49.167 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.167 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.167 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.167 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:49.168 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.168 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.168 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.168 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.169 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.169 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.169 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.169 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.170 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T05:00:49.170 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T05:00:49.170 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.170 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.170 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.171 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.171 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.171 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.171 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.171 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.172 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.172 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.172 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.173 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T05:00:49.173 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T05:00:49.173 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.174 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.174 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.174 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.174 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.174 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.175 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.175 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.175 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.175 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.175 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.176 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T05:00:49.176 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T05:00:49.177 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.177 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.177 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.177 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:49.177 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.178 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.178 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.178 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.178 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.178 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.179 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.179 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.179 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T05:00:49.179 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T05:00:49.179 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.180 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.180 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.180 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:49.180 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.180 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.181 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.181 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.181 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.181 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.181 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.182 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.182 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T05:00:49.182 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T05:00:49.182 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.182 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.183 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.184 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.184 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.184 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.184 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.184 INFO:teuthology.orchestra.run.smithi145.stdout: "cross_origin_url": { 2023-08-25T05:00:49.185 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cross_origin_url", 2023-08-25T05:00:49.185 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.185 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.185 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.186 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.187 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.187 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.187 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.187 INFO:teuthology.orchestra.run.smithi145.stdout: "crt_file": { 2023-08-25T05:00:49.187 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crt_file", 2023-08-25T05:00:49.188 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.188 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.188 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.188 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.188 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.189 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.189 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.189 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.189 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.190 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.190 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.190 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.190 INFO:teuthology.orchestra.run.smithi145.stdout: "debug": { 2023-08-25T05:00:49.190 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "debug", 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable/disable debug options", 2023-08-25T05:00:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.193 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.193 INFO:teuthology.orchestra.run.smithi145.stdout: "jwt_token_ttl": { 2023-08-25T05:00:49.193 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "jwt_token_ttl", 2023-08-25T05:00:49.193 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.193 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.194 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.194 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "28800", 2023-08-25T05:00:49.194 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.194 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.194 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.195 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.195 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.195 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.195 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.196 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:49.196 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:49.196 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.196 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.196 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.197 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.197 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.197 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.198 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.198 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.198 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.198 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.198 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.199 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.199 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.199 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.199 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.199 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.200 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.201 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.201 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.201 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.201 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.202 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.202 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.202 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.202 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.203 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.203 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.203 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.203 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.203 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.204 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.204 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.204 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.204 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.204 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.205 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.206 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.206 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.206 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.206 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.207 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.207 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.207 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.207 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.207 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.208 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.208 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.208 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.208 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.208 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.209 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.209 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.209 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.209 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.209 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.210 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.210 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.210 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.210 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.210 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.211 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.211 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.211 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.211 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.211 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.212 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.213 INFO:teuthology.orchestra.run.smithi145.stdout: "motd": { 2023-08-25T05:00:49.213 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "motd", 2023-08-25T05:00:49.213 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.213 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.213 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.214 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.214 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.214 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.214 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.214 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "The message of the day", 2023-08-25T05:00:49.215 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.215 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.215 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.215 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect_resolve_ip_addr": { 2023-08-25T05:00:49.216 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T05:00:49.216 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.216 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.216 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.216 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.217 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.217 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.217 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.217 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.217 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.218 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.218 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.218 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.218 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:49.218 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.219 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.220 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.220 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.220 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.220 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.220 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.221 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.221 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:49.221 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:49.221 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.222 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.222 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.222 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8080", 2023-08-25T05:00:49.222 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.223 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.224 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:49.224 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:49.224 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.224 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.225 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.225 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.225 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.225 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.225 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.227 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl_server_port": { 2023-08-25T05:00:49.227 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl_server_port", 2023-08-25T05:00:49.227 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.227 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.228 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.228 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8443", 2023-08-25T05:00:49.228 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.228 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.228 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.229 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.229 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.229 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.229 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.230 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:49.230 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:49.230 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.230 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.230 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.231 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "redirect", 2023-08-25T05:00:49.231 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.231 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.231 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.232 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.232 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect" 2023-08-25T05:00:49.232 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.232 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.232 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.233 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.233 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.233 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.234 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:49.234 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:49.234 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.234 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.235 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.235 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:49.235 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:49.235 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:49.235 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.236 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.236 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.236 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.236 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.237 INFO:teuthology.orchestra.run.smithi145.stdout: "url_prefix": { 2023-08-25T05:00:49.237 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url_prefix", 2023-08-25T05:00:49.237 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.237 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.237 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.238 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.239 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.239 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.239 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.239 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.239 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.240 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.240 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "devicehealth", 2023-08-25T05:00:49.240 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.240 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.241 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.241 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_monitoring": { 2023-08-25T05:00:49.241 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_monitoring", 2023-08-25T05:00:49.241 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.242 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.242 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.242 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.242 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.242 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.243 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.243 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "monitor device health metrics", 2023-08-25T05:00:49.243 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.243 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.243 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.244 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.244 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.245 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.245 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.245 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.245 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.246 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.246 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.246 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.247 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.247 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.247 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.247 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.248 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.248 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.249 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.249 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.250 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.250 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.250 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.251 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.251 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.251 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.251 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.252 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.252 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.253 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.253 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.253 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.254 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.254 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.254 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.255 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.255 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.255 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.256 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.256 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.256 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.257 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.257 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.257 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.257 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.257 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.258 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.258 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.258 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.258 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.259 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.259 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.259 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.259 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.260 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.260 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.260 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.261 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.261 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.262 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.262 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.262 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.262 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.263 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.263 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.263 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.263 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.264 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.264 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.264 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.265 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.265 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.265 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.265 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.265 INFO:teuthology.orchestra.run.smithi145.stdout: "mark_out_threshold": { 2023-08-25T05:00:49.266 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mark_out_threshold", 2023-08-25T05:00:49.266 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.266 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.266 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.267 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2419200", 2023-08-25T05:00:49.267 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.267 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.267 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.268 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T05:00:49.268 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.268 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.268 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.269 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_name": { 2023-08-25T05:00:49.269 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_name", 2023-08-25T05:00:49.269 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.269 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.270 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.270 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "device_health_metrics", 2023-08-25T05:00:49.270 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.271 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.271 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.272 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T05:00:49.272 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.272 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.273 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.274 INFO:teuthology.orchestra.run.smithi145.stdout: "retention_period": { 2023-08-25T05:00:49.274 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retention_period", 2023-08-25T05:00:49.274 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.275 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.275 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.275 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15552000", 2023-08-25T05:00:49.276 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.276 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.276 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.277 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain device health metrics", 2023-08-25T05:00:49.277 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.277 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.278 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.278 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.278 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_frequency": { 2023-08-25T05:00:49.279 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_frequency", 2023-08-25T05:00:49.279 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.279 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.279 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.280 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:49.280 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.280 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.280 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.281 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T05:00:49.281 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.281 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.281 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.282 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.282 INFO:teuthology.orchestra.run.smithi145.stdout: "self_heal": { 2023-08-25T05:00:49.282 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "self_heal", 2023-08-25T05:00:49.282 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.282 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.283 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.283 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.283 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.283 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.284 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.284 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T05:00:49.284 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.284 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.284 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.285 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.285 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:49.285 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:49.286 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.286 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.286 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.286 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:49.287 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.287 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.287 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.288 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T05:00:49.288 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.288 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.289 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.289 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.289 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_threshold": { 2023-08-25T05:00:49.290 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_threshold", 2023-08-25T05:00:49.290 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.290 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.290 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.291 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7257600", 2023-08-25T05:00:49.291 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.291 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.291 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.292 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T05:00:49.292 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.292 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.293 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.293 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.293 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.294 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.294 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.294 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "diskprediction_local", 2023-08-25T05:00:49.294 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.295 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.295 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.295 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.296 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.296 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.296 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.297 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.297 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.297 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.298 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.298 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.298 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.299 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.299 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.299 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.299 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.300 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.300 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.300 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.301 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.301 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.301 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.302 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.302 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.302 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.303 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.303 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.303 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.304 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.304 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.304 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.304 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.305 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.305 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.305 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.306 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.306 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.306 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.306 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.306 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.307 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.307 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.307 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.307 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.308 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.308 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.309 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.309 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.309 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.309 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.310 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.310 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.310 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.310 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.311 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.311 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.311 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.311 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.312 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.312 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.313 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.313 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.313 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.313 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.313 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.314 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.314 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.314 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.314 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.315 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.315 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.316 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.316 INFO:teuthology.orchestra.run.smithi145.stdout: "predict_interval": { 2023-08-25T05:00:49.316 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predict_interval", 2023-08-25T05:00:49.317 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.317 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.318 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.319 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.319 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.319 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.319 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.319 INFO:teuthology.orchestra.run.smithi145.stdout: "predictor_model": { 2023-08-25T05:00:49.320 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predictor_model", 2023-08-25T05:00:49.320 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.320 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.320 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.321 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "prophetstor", 2023-08-25T05:00:49.321 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.321 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.321 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.321 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.322 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.322 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.322 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.322 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:49.323 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:49.323 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.323 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.323 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.323 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:49.324 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.324 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.324 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.324 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.324 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.325 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.325 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.325 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.325 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.325 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "influx", 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": false, 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "influxdb python module not found", 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.326 INFO:teuthology.orchestra.run.smithi145.stdout: "batch_size": { 2023-08-25T05:00:49.327 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "batch_size", 2023-08-25T05:00:49.327 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.327 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.327 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.327 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5000", 2023-08-25T05:00:49.328 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.328 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.328 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.328 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T05:00:49.328 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.329 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.329 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.329 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.329 INFO:teuthology.orchestra.run.smithi145.stdout: "database": { 2023-08-25T05:00:49.329 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "database", 2023-08-25T05:00:49.330 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.330 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.330 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.330 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "ceph", 2023-08-25T05:00:49.330 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.331 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.331 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.331 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T05:00:49.331 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.331 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": { 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "hostname", 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.332 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.333 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.333 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.333 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.333 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server hostname", 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.335 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:49.335 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:49.335 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.335 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.335 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.336 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "30", 2023-08-25T05:00:49.336 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "5", 2023-08-25T05:00:49.336 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.336 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.336 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T05:00:49.337 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.337 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.337 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.337 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.337 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.338 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.338 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.338 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.338 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.338 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.339 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.339 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.339 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.339 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.339 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.340 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.340 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.340 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.340 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.340 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.341 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.341 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.341 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.341 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.341 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.342 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.342 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.342 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.342 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.342 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.343 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.343 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.343 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.343 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.343 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.344 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.344 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.344 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.344 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.344 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.346 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.346 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.346 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.346 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.346 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.347 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.347 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.347 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.347 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.347 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.348 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.348 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.348 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.348 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.349 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.349 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.349 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.349 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.349 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.350 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.350 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.350 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.350 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.350 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.351 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.351 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.351 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.351 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.351 INFO:teuthology.orchestra.run.smithi145.stdout: "password": { 2023-08-25T05:00:49.352 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "password", 2023-08-25T05:00:49.352 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.352 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.352 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.352 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "password of InfluxDB server user", 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.353 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.354 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.354 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.354 INFO:teuthology.orchestra.run.smithi145.stdout: "port": { 2023-08-25T05:00:49.354 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "port", 2023-08-25T05:00:49.354 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.355 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.355 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.355 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8086", 2023-08-25T05:00:49.355 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.355 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.356 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.356 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server port", 2023-08-25T05:00:49.356 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.356 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.356 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.357 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:49.357 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:49.357 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.357 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.358 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.358 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "false", 2023-08-25T05:00:49.358 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.358 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.358 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.359 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "threads": { 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threads", 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.360 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "32", 2023-08-25T05:00:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T05:00:49.361 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "username": { 2023-08-25T05:00:49.362 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "username", 2023-08-25T05:00:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.363 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "username of InfluxDB server user", 2023-08-25T05:00:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.364 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "verify_ssl": { 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "verify_ssl", 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.365 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "true", 2023-08-25T05:00:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.366 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.367 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "insights", 2023-08-25T05:00:49.368 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.369 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.370 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.372 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.373 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.374 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.375 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.376 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.377 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.378 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.379 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.380 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.381 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.382 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.383 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "iostat", 2023-08-25T05:00:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.384 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.385 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.386 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.387 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.388 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.389 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.390 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.391 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.392 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.393 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.394 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.395 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.397 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.398 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "k8sevents", 2023-08-25T05:00:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_event_retention_days": { 2023-08-25T05:00:49.399 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ceph_event_retention_days", 2023-08-25T05:00:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7", 2023-08-25T05:00:49.400 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T05:00:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.401 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "config_check_secs": { 2023-08-25T05:00:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_check_secs", 2023-08-25T05:00:49.402 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "10", 2023-08-25T05:00:49.403 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T05:00:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.404 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.405 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.406 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.407 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.408 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.409 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.410 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.411 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.412 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.413 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.414 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.415 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.416 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.417 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.418 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.419 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "localpool", 2023-08-25T05:00:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.420 INFO:teuthology.orchestra.run.smithi145.stdout: "failure_domain": { 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "failure_domain", 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "host", 2023-08-25T05:00:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "failure domain for any created local pool", 2023-08-25T05:00:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T05:00:49.422 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.423 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.424 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.425 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.426 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.427 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.428 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.429 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.430 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.431 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.432 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.433 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.434 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.435 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.436 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "min_size": { 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_size", 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.437 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.438 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default min_size for any created local pool", 2023-08-25T05:00:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T05:00:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.439 INFO:teuthology.orchestra.run.smithi145.stdout: "num_rep": { 2023-08-25T05:00:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "num_rep", 2023-08-25T05:00:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.440 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3", 2023-08-25T05:00:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default replica count for any created local pool", 2023-08-25T05:00:49.441 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_num": { 2023-08-25T05:00:49.442 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_num", 2023-08-25T05:00:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "128", 2023-08-25T05:00:49.443 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T05:00:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.444 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "prefix": { 2023-08-25T05:00:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prefix", 2023-08-25T05:00:49.445 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name prefix for any created local pool", 2023-08-25T05:00:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.447 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "subtree": { 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "subtree", 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.448 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "rack", 2023-08-25T05:00:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.449 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T05:00:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T05:00:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.450 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mds_autoscaler", 2023-08-25T05:00:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.451 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.452 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.454 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.455 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.456 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.457 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.458 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.459 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.460 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.461 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.463 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.464 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.465 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirroring", 2023-08-25T05:00:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.466 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.467 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.468 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.469 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.470 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.471 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.472 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.474 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.475 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.476 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.478 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.479 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.480 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "nfs", 2023-08-25T05:00:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.481 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.482 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.483 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.484 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.485 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.486 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.487 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.488 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.489 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.490 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.491 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.492 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.493 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.494 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.495 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.496 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "fail_fs": { 2023-08-25T05:00:49.497 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "fail_fs", 2023-08-25T05:00:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.498 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.499 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.500 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.501 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.502 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.503 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.504 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.505 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.506 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.507 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.508 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.509 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.510 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.511 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.512 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.513 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator": { 2023-08-25T05:00:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.514 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm", 2023-08-25T05:00:49.515 INFO:teuthology.orchestra.run.smithi145.stdout: "rook", 2023-08-25T05:00:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "test_orchestrator" 2023-08-25T05:00:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Orchestrator backend", 2023-08-25T05:00:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.516 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.517 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_perf_query", 2023-08-25T05:00:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.518 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.519 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.520 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.521 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.522 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.523 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.524 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.525 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.526 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.527 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.528 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.529 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.530 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.531 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.532 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_support", 2023-08-25T05:00:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.533 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.534 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.535 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.536 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.537 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.538 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.539 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.540 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.541 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.542 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.543 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.544 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.545 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.546 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_autoscaler", 2023-08-25T05:00:49.547 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.548 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.549 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.550 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.551 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.552 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.553 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.554 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.555 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.556 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.557 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.558 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.559 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.560 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.561 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.561 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.561 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.561 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.561 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.562 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.562 INFO:teuthology.orchestra.run.smithi145.stdout: "noautoscale": { 2023-08-25T05:00:49.562 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "noautoscale", 2023-08-25T05:00:49.562 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.562 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.563 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.563 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.563 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.563 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.563 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.564 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "global autoscale flag", 2023-08-25T05:00:49.564 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T05:00:49.564 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.564 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.564 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.565 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:49.565 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:49.565 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.565 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.565 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.566 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:49.566 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.566 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.566 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.566 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.567 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.567 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.567 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.567 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.567 INFO:teuthology.orchestra.run.smithi145.stdout: "threshold": { 2023-08-25T05:00:49.568 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threshold", 2023-08-25T05:00:49.568 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:49.568 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.568 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.568 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1.0", 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "scaling threshold", 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T05:00:49.569 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.570 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.570 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.570 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.570 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.570 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.571 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "progress", 2023-08-25T05:00:49.571 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.571 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.571 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.571 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_pg_recovery_event": { 2023-08-25T05:00:49.572 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_pg_recovery_event", 2023-08-25T05:00:49.572 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.572 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.572 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.572 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.573 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.573 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.573 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.573 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T05:00:49.573 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:49.574 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.575 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.575 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.575 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.575 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.575 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.576 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.576 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.576 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.576 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.576 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.577 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.577 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.577 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.577 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.577 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.578 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.578 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.578 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.578 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.578 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.579 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.579 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.579 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.579 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.579 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.580 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.580 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.580 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.580 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.580 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.581 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.581 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.581 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.581 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.581 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.582 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.582 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.582 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.582 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.582 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.586 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.586 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.586 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.586 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.586 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.587 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.588 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.588 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.588 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.588 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.588 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.589 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.589 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.589 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.589 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.589 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.590 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.590 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.590 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.590 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.590 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.591 INFO:teuthology.orchestra.run.smithi145.stdout: "max_completed_events": { 2023-08-25T05:00:49.591 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_completed_events", 2023-08-25T05:00:49.591 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.591 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.591 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.592 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "50", 2023-08-25T05:00:49.592 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.592 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.592 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.592 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "number of past completed events to remember", 2023-08-25T05:00:49.593 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.593 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.593 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.593 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.593 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:49.594 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.595 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.595 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.595 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long the module is going to sleep", 2023-08-25T05:00:49.595 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.595 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.596 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.596 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.596 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.596 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.596 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.597 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus", 2023-08-25T05:00:49.597 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.597 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.597 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.598 INFO:teuthology.orchestra.run.smithi145.stdout: "cache": { 2023-08-25T05:00:49.598 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cache", 2023-08-25T05:00:49.598 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.598 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.598 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.599 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.600 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.600 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.600 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.600 INFO:teuthology.orchestra.run.smithi145.stdout: "exclude_perf_counters": { 2023-08-25T05:00:49.600 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "exclude_perf_counters", 2023-08-25T05:00:49.601 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.601 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.601 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.601 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.601 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.602 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.602 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.602 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T05:00:49.602 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T05:00:49.602 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.603 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.603 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.603 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.603 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.603 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.604 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.604 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.604 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.604 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.604 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.605 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.605 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.605 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.605 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.605 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.606 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.606 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.606 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.606 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.606 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.607 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.607 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.607 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.607 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.608 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.608 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.608 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.608 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.608 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.609 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.609 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.609 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.609 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.609 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.610 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.610 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.610 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.610 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.611 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.611 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.611 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.611 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.611 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.612 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.612 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.612 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.612 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.612 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.613 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.613 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.613 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.613 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.613 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.614 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.614 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.614 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.614 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.614 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.615 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.615 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.615 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.615 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.615 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.616 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.616 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.616 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.616 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.616 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.617 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.617 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.617 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.617 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools": { 2023-08-25T05:00:49.617 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools", 2023-08-25T05:00:49.618 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.618 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.618 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.618 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.618 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.619 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.619 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.619 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.619 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.619 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.620 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.620 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.620 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T05:00:49.620 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T05:00:49.620 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.621 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.621 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.621 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300", 2023-08-25T05:00:49.621 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.621 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.622 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.623 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_interval": { 2023-08-25T05:00:49.623 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_interval", 2023-08-25T05:00:49.623 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:49.623 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.623 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.624 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15.0", 2023-08-25T05:00:49.624 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.624 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.624 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.624 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.625 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.625 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.625 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.625 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.625 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:49.626 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.627 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.627 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.627 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T05:00:49.627 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.627 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.628 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.628 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.628 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:49.628 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:49.628 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.629 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.629 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.629 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "9283", 2023-08-25T05:00:49.629 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.629 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.630 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.631 INFO:teuthology.orchestra.run.smithi145.stdout: "stale_cache_strategy": { 2023-08-25T05:00:49.631 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stale_cache_strategy", 2023-08-25T05:00:49.631 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.631 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.631 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.632 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "log", 2023-08-25T05:00:49.632 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.632 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.632 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.632 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.633 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.633 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.633 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.633 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.634 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "default", 2023-08-25T05:00:49.635 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.635 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.635 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.635 INFO:teuthology.orchestra.run.smithi145.stdout: "default", 2023-08-25T05:00:49.635 INFO:teuthology.orchestra.run.smithi145.stdout: "error" 2023-08-25T05:00:49.636 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.636 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.636 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.636 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.636 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.637 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:49.637 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:49.637 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.637 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.638 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.638 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:49.638 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:49.638 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:49.638 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.639 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.640 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.640 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.640 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_support", 2023-08-25T05:00:49.640 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.640 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.642 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.642 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.642 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.642 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.642 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.643 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.643 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.643 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.643 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.643 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.644 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.644 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.644 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.644 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.644 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.645 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.645 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.645 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.645 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.647 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.647 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.647 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.647 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.647 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.648 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.648 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.648 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.648 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.648 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.649 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.649 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.649 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.649 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.649 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.650 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.651 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.651 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.651 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.651 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.651 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.652 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.652 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.652 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.652 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.652 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.653 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.653 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.653 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.653 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.653 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_snap_create": { 2023-08-25T05:00:49.654 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_snap_create", 2023-08-25T05:00:49.655 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.655 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.655 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.655 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:49.655 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.656 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.656 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.656 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.656 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.656 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.657 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.657 INFO:teuthology.orchestra.run.smithi145.stdout: "mirror_snapshot_schedule": { 2023-08-25T05:00:49.657 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T05:00:49.657 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.658 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.658 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.658 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.658 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.658 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.660 INFO:teuthology.orchestra.run.smithi145.stdout: "trash_purge_schedule": { 2023-08-25T05:00:49.660 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "trash_purge_schedule", 2023-08-25T05:00:49.660 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.660 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.660 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.661 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.661 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.661 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.661 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.661 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.662 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.662 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.662 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.662 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.662 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.663 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.663 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.663 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "restful", 2023-08-25T05:00:49.663 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.663 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.664 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.664 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_auth": { 2023-08-25T05:00:49.664 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_auth", 2023-08-25T05:00:49.664 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.664 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.665 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.665 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.665 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.665 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.665 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.666 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.666 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.666 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.666 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.667 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.668 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.668 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.668 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.668 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.668 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.669 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.669 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.669 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.669 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.669 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.670 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.670 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.670 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.670 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.670 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.671 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.671 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.671 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.671 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.671 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.672 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.672 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.672 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.672 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.672 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.673 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.673 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.673 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.673 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.674 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.674 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.674 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.674 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.674 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.675 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.675 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.675 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.675 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.675 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.676 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.676 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.676 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.676 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.677 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.678 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.678 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.678 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.678 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.678 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.679 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.679 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.679 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.679 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.679 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.680 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.680 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.680 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.680 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.680 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.681 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.681 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.681 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.681 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.681 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.682 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.683 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:49.683 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:49.683 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.683 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.684 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.684 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.684 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.684 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.684 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.685 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.685 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.685 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.685 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.685 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.686 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:49.686 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:49.686 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.686 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.686 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.687 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.688 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.688 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.688 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.689 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.689 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rgw", 2023-08-25T05:00:49.689 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.689 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.689 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.690 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.690 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.690 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.690 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.690 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.691 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.691 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.691 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.691 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.691 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.692 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.692 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.692 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.692 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.692 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.694 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.694 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.694 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.694 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.694 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.695 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.695 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.695 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.695 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.695 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.696 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.696 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.696 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.696 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.696 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.697 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.697 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.697 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.697 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.697 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.698 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.698 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.698 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.698 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.698 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.699 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.699 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.699 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.699 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.699 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.700 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.700 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.700 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.700 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.700 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.701 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.701 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.701 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.701 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.701 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.702 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.703 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.703 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.703 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.703 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.704 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.704 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rook", 2023-08-25T05:00:49.704 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.704 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.704 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.705 INFO:teuthology.orchestra.run.smithi145.stdout: "drive_group_interval": { 2023-08-25T05:00:49.705 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "drive_group_interval", 2023-08-25T05:00:49.705 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:49.705 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.705 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.706 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300.0", 2023-08-25T05:00:49.706 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.706 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.706 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.706 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T05:00:49.707 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.707 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.707 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.707 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.707 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.708 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.708 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.708 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.708 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.708 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.709 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.709 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.709 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.709 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.709 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.710 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.710 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.710 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.710 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.710 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.711 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.712 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.712 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.712 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.712 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.712 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.713 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.713 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.713 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.713 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.713 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.714 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.714 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.714 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.714 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.714 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.715 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.715 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.715 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.715 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.715 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.716 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.716 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.716 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.716 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.716 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.717 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.717 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.717 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.717 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.717 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.718 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.718 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.718 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.718 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.718 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.719 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.720 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.720 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.720 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.720 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.720 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.721 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.721 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.721 INFO:teuthology.orchestra.run.smithi145.stdout: "storage_class": { 2023-08-25T05:00:49.721 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "storage_class", 2023-08-25T05:00:49.721 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.722 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.722 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.722 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "local", 2023-08-25T05:00:49.722 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.722 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.723 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.723 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T05:00:49.723 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.723 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.723 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "selftest", 2023-08-25T05:00:49.725 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.725 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.725 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.725 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.725 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.726 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.726 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.726 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.726 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.726 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.727 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.728 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.728 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.728 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.728 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.728 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.729 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.729 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.729 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.729 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.730 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.730 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.730 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.730 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.730 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.731 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.731 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.731 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.731 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.731 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.732 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.732 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.732 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.732 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.733 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.733 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.733 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.733 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.733 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.734 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.735 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.735 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.735 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.735 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.735 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.736 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.736 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.736 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.736 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.737 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.737 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.737 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.737 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.737 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.738 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.738 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.738 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.738 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.738 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: "roption1": { 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption1", 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.739 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.740 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.740 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.740 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.740 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.741 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.741 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.741 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.742 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.742 INFO:teuthology.orchestra.run.smithi145.stdout: "roption2": { 2023-08-25T05:00:49.742 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption2", 2023-08-25T05:00:49.742 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.743 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.743 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.743 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "xyz", 2023-08-25T05:00:49.743 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.743 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.744 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.744 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.744 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.744 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.744 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.745 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption1": { 2023-08-25T05:00:49.745 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption1", 2023-08-25T05:00:49.745 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.745 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.746 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.746 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.746 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.746 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.746 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.747 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.747 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.747 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.747 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.748 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption2": { 2023-08-25T05:00:49.748 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption2", 2023-08-25T05:00:49.748 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.748 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.748 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.749 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.749 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.749 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.749 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.749 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.750 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.750 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.750 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.750 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption3": { 2023-08-25T05:00:49.751 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption3", 2023-08-25T05:00:49.751 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:49.751 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.751 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.751 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.752 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.752 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.752 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.752 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.752 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.753 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.753 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.753 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.753 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption4": { 2023-08-25T05:00:49.753 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption4", 2023-08-25T05:00:49.754 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.754 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.754 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.754 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.754 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.756 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption5": { 2023-08-25T05:00:49.756 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption5", 2023-08-25T05:00:49.756 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.756 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.757 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.757 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.757 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.757 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.757 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.758 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.758 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.758 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.758 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.759 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption6": { 2023-08-25T05:00:49.759 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption6", 2023-08-25T05:00:49.759 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.759 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.759 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.760 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.760 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.760 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.760 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.760 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption7": { 2023-08-25T05:00:49.761 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption7", 2023-08-25T05:00:49.762 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.762 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.762 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.763 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.763 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:49.763 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "42", 2023-08-25T05:00:49.763 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.764 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.764 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.764 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.764 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.764 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.765 INFO:teuthology.orchestra.run.smithi145.stdout: "testkey": { 2023-08-25T05:00:49.765 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testkey", 2023-08-25T05:00:49.765 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.765 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.765 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.766 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.766 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.766 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.766 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.766 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: "testlkey": { 2023-08-25T05:00:49.767 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testlkey", 2023-08-25T05:00:49.768 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.768 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.768 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.768 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.768 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.769 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: "testnewline": { 2023-08-25T05:00:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testnewline", 2023-08-25T05:00:49.770 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.771 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.772 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.773 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.773 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.773 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.773 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.773 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snap_schedule", 2023-08-25T05:00:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_m_granularity": { 2023-08-25T05:00:49.774 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_m_granularity", 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.775 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.776 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.776 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T05:00:49.776 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.776 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.776 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.777 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.777 INFO:teuthology.orchestra.run.smithi145.stdout: "dump_on_update": { 2023-08-25T05:00:49.777 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dump_on_update", 2023-08-25T05:00:49.777 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.777 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.778 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.778 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.778 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.778 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.778 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.779 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "dump database to debug log on update", 2023-08-25T05:00:49.779 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.779 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.779 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.780 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.780 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.780 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.780 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.780 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.781 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.781 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.781 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.781 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.781 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.783 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.783 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.783 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.784 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.784 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.784 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.784 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.785 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.785 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.785 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.785 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.785 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.786 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.786 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.786 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.786 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.786 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.787 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.788 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.788 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.788 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.788 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.788 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.789 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.789 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.790 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.790 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.790 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.790 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.790 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.791 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.791 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.791 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.791 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.792 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.792 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.792 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.792 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.792 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.793 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.793 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.793 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.793 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.793 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.795 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.795 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stats", 2023-08-25T05:00:49.795 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.795 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.795 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.796 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.796 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.796 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.796 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.796 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.797 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.797 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.797 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.797 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.797 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.798 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.798 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.798 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.798 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.798 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.799 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.799 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.799 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.799 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.799 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.800 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.800 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.800 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.800 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.801 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.801 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.801 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.801 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.801 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.802 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.802 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.802 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.802 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.803 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.804 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.804 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.804 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.804 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.804 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.805 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.805 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.805 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.805 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.805 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.806 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.806 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.806 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.806 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.807 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.807 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.807 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.807 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.807 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.809 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.809 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.809 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.809 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.809 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.810 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.810 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.810 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "status", 2023-08-25T05:00:49.810 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.810 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.811 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.812 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.812 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.812 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.813 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.813 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.813 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.813 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.814 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.815 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.815 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.815 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.815 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.815 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.816 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.816 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.816 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.816 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.816 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.817 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.817 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.817 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.817 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.817 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.818 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.818 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.818 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.818 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.818 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.819 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.819 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.819 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.819 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.819 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.820 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.820 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.820 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.820 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.820 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.821 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.822 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.822 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.822 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.822 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.823 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.823 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.823 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.823 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.823 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.824 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.824 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.824 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.824 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.824 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.825 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.825 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.825 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.825 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.825 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telegraf", 2023-08-25T05:00:49.826 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.826 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.826 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.826 INFO:teuthology.orchestra.run.smithi145.stdout: "address": { 2023-08-25T05:00:49.826 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "address", 2023-08-25T05:00:49.827 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.827 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.827 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.827 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T05:00:49.827 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.828 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.828 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.828 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.829 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.829 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.829 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.829 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:49.830 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:49.830 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.830 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.830 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.830 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15", 2023-08-25T05:00:49.831 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.831 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.831 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.831 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.831 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.832 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.832 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.832 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.832 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.832 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.834 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.834 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.834 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.834 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.834 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.835 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.835 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.835 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.835 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.835 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.836 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.836 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.836 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.836 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.836 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.837 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.837 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.837 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.838 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.838 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.838 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.838 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.838 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.839 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.839 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.839 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.839 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.839 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.840 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.840 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.840 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.840 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.840 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.841 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.841 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.841 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.841 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.841 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.842 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.842 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.842 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.842 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.842 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.843 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.844 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.844 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.844 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.844 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.844 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.845 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.845 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.845 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.845 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.845 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.846 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.846 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.846 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.846 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.846 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.847 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.847 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telemetry", 2023-08-25T05:00:49.847 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.847 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.847 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.848 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_basic": { 2023-08-25T05:00:49.848 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_basic", 2023-08-25T05:00:49.848 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.848 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.848 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T05:00:49.849 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_crash": { 2023-08-25T05:00:49.850 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_crash", 2023-08-25T05:00:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.851 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.852 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.852 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.852 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T05:00:49.852 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.852 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_device": { 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_device", 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.853 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.854 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.854 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:49.854 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.854 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.854 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.855 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T05:00:49.855 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.855 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.855 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.855 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.856 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_ident": { 2023-08-25T05:00:49.856 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_ident", 2023-08-25T05:00:49.856 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.856 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.856 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.857 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.857 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.857 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.857 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.857 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_perf": { 2023-08-25T05:00:49.858 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_perf", 2023-08-25T05:00:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T05:00:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: "contact": { 2023-08-25T05:00:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "contact", 2023-08-25T05:00:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.863 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.863 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.863 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.863 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: "description": { 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "description", 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.864 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.866 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.867 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.867 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.868 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.868 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.868 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.869 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.869 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.870 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.870 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.870 INFO:teuthology.orchestra.run.smithi145.stdout: "device_url": { 2023-08-25T05:00:49.871 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_url", 2023-08-25T05:00:49.871 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.871 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.871 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.872 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T05:00:49.872 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.872 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.872 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.873 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.873 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.873 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.873 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.873 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.874 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:49.874 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:49.874 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.874 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.875 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.875 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.876 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.876 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.876 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.877 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.877 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.877 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.877 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.878 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.878 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:49.879 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:49.879 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.879 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.879 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.880 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "24", 2023-08-25T05:00:49.880 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "8", 2023-08-25T05:00:49.880 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.881 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.881 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.881 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.882 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.882 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.882 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.883 INFO:teuthology.orchestra.run.smithi145.stdout: "last_opt_revision": { 2023-08-25T05:00:49.883 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "last_opt_revision", 2023-08-25T05:00:49.883 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.884 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.884 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.885 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1", 2023-08-25T05:00:49.885 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.885 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.886 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.886 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.886 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.887 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.887 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.887 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard": { 2023-08-25T05:00:49.888 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard", 2023-08-25T05:00:49.888 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.888 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.889 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.889 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.889 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.890 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.890 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.891 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.891 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.891 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.892 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.892 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.892 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard_description": { 2023-08-25T05:00:49.893 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard_description", 2023-08-25T05:00:49.893 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.893 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.894 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.894 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.895 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.895 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.895 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.896 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.896 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.897 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.897 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.898 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.899 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.899 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.899 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.900 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.900 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.901 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.901 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.902 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.902 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.903 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.903 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.904 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.904 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.904 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.905 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.905 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.906 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.906 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.906 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.907 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.907 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.907 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.908 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.908 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.909 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.909 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.910 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.910 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.910 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.911 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.911 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.911 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.912 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.912 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.912 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.912 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.913 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.913 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.913 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.914 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.914 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.914 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.914 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.914 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.915 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.915 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.915 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.915 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.916 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.916 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.916 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.916 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.917 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.917 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.918 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.918 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.919 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.919 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.919 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.920 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.920 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.921 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.921 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.921 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.922 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.922 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.922 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.923 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.923 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.923 INFO:teuthology.orchestra.run.smithi145.stdout: "organization": { 2023-08-25T05:00:49.924 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "organization", 2023-08-25T05:00:49.924 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.925 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.925 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.925 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.926 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.926 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.926 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.926 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.927 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.927 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.928 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.928 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.929 INFO:teuthology.orchestra.run.smithi145.stdout: "proxy": { 2023-08-25T05:00:49.929 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "proxy", 2023-08-25T05:00:49.930 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.930 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.930 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.931 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.931 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.932 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.932 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.933 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.933 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.934 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.934 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.934 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.935 INFO:teuthology.orchestra.run.smithi145.stdout: "url": { 2023-08-25T05:00:49.935 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url", 2023-08-25T05:00:49.935 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.935 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.936 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.936 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T05:00:49.936 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.937 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.937 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.937 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.938 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.938 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.938 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.939 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.939 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.939 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.940 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.940 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "test_orchestrator", 2023-08-25T05:00:49.940 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.941 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.941 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.941 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.942 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.942 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.942 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.942 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.942 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.943 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.943 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.943 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.943 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.943 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.944 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.944 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.944 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.945 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.945 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.945 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.946 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.946 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.946 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.947 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.947 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.947 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.947 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.947 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.948 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.948 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.948 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.949 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.949 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.949 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.949 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.949 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.950 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.951 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.951 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.951 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.951 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.951 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.952 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.952 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.952 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.952 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.953 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.953 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.953 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.953 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.954 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.954 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.954 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.955 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.955 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.956 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.956 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.957 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.957 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.957 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.958 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.958 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.958 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.958 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.959 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.959 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.959 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.960 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.961 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.961 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "volumes", 2023-08-25T05:00:49.961 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.962 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.962 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.962 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.963 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.963 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.963 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.964 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.964 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.965 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.965 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.965 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.966 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.966 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.966 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.966 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.966 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.967 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.967 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.967 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.967 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.967 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.968 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.968 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.968 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.968 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.968 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.969 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.970 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.970 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.970 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.970 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.971 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.971 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.971 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.971 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.971 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.972 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:49.972 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.972 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.972 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.972 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.973 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.973 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.973 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.973 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.973 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.974 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.974 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.974 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.974 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.974 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.975 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.976 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.976 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.976 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.976 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.976 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.977 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.977 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.977 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.977 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_clones": { 2023-08-25T05:00:49.978 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_clones", 2023-08-25T05:00:49.978 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.978 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.978 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.978 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4", 2023-08-25T05:00:49.979 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.979 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.979 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.979 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T05:00:49.979 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.980 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.980 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.980 INFO:teuthology.orchestra.run.smithi145.stdout: "snapshot_clone_delay": { 2023-08-25T05:00:49.980 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snapshot_clone_delay", 2023-08-25T05:00:49.981 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:49.981 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.981 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.981 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:49.981 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.982 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.983 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.983 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:49.983 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.983 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:49.983 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix", 2023-08-25T05:00:49.984 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:49.984 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:49.984 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:49.984 INFO:teuthology.orchestra.run.smithi145.stdout: "discovery_interval": { 2023-08-25T05:00:49.984 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "discovery_interval", 2023-08-25T05:00:49.985 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:49.985 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.985 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.985 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "100", 2023-08-25T05:00:49.985 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.986 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.987 INFO:teuthology.orchestra.run.smithi145.stdout: "identifier": { 2023-08-25T05:00:49.987 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "identifier", 2023-08-25T05:00:49.987 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.987 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.988 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.988 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.988 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.988 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.988 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.989 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.989 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.989 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.989 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.989 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.990 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:49.990 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:49.990 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:49.990 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.990 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.991 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.992 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.992 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.992 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:49.992 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:49.993 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:49.993 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.993 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.993 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:49.993 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.994 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.994 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:49.994 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:49.994 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:49.994 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:49.995 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:49.995 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:49.995 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:49.995 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:49.995 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.996 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.996 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.996 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.996 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.996 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:49.997 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:49.998 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:49.998 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:49.998 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:49.998 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:49.998 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:49.999 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:49.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:49.999 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:49.999 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:49.999 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.000 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.001 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.001 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.001 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.001 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.001 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.002 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.002 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.002 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.002 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.002 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.003 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.003 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.003 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.003 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.003 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.004 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.004 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.004 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.004 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.004 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.006 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_host": { 2023-08-25T05:00:50.006 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_host", 2023-08-25T05:00:50.006 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.006 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.006 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.007 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.007 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.007 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.007 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.007 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.008 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.008 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.008 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.008 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.008 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_port": { 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_port", 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10051", 2023-08-25T05:00:50.009 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.010 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.010 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.010 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.010 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.010 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.011 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.011 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.011 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_sender": { 2023-08-25T05:00:50.011 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_sender", 2023-08-25T05:00:50.011 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.012 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.012 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.012 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T05:00:50.012 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.012 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.013 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.014 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.014 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.014 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:50.014 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.014 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.015 INFO:teuthology.orchestra.run.smithi145.stdout: "modules": [ 2023-08-25T05:00:50.015 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm", 2023-08-25T05:00:50.015 INFO:teuthology.orchestra.run.smithi145.stdout: "dashboard", 2023-08-25T05:00:50.015 INFO:teuthology.orchestra.run.smithi145.stdout: "iostat", 2023-08-25T05:00:50.015 INFO:teuthology.orchestra.run.smithi145.stdout: "nfs", 2023-08-25T05:00:50.016 INFO:teuthology.orchestra.run.smithi145.stdout: "restful", 2023-08-25T05:00:50.016 INFO:teuthology.orchestra.run.smithi145.stdout: "snap_schedule" 2023-08-25T05:00:50.016 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.016 INFO:teuthology.orchestra.run.smithi145.stdout: "available_modules": [ 2023-08-25T05:00:50.016 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "alerts", 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:50.017 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:50.018 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.018 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.018 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.018 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:50.018 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.019 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.019 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.019 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How frequently to reexamine health status", 2023-08-25T05:00:50.019 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.019 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.020 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.020 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.020 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.020 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.021 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.021 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.021 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.021 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.021 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.022 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.022 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.022 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.022 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.022 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.023 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.024 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.024 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.024 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.024 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.025 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.025 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.025 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.025 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.025 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.026 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.026 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.026 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.026 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.026 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.027 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.027 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.027 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.027 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.027 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.028 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.028 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.028 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.028 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.029 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.029 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.029 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.029 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.029 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.030 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.030 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.030 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.030 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.030 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.031 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.031 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.031 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.031 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.032 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.032 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.032 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.032 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.032 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.033 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.034 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_destination": { 2023-08-25T05:00:50.034 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_destination", 2023-08-25T05:00:50.034 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.034 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.035 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.035 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.035 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.035 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.035 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.036 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email address to send alerts to", 2023-08-25T05:00:50.036 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.036 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.036 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_from_name": { 2023-08-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_from_name", 2023-08-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "Ceph", 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Email From: name", 2023-08-25T05:00:50.038 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.039 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.039 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.039 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_host": { 2023-08-25T05:00:50.039 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_host", 2023-08-25T05:00:50.040 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.040 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.040 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.040 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.040 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.041 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.041 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.041 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP server", 2023-08-25T05:00:50.041 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.041 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_password": { 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_password", 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.042 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.043 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.043 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.043 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.043 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.043 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.044 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Password to authenticate with", 2023-08-25T05:00:50.044 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.044 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.044 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.045 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_port": { 2023-08-25T05:00:50.045 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_port", 2023-08-25T05:00:50.045 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.045 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.045 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "465", 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP port", 2023-08-25T05:00:50.046 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.047 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.047 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.047 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_sender": { 2023-08-25T05:00:50.047 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_sender", 2023-08-25T05:00:50.048 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.048 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.048 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.048 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.048 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.049 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.049 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.049 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SMTP envelope sender", 2023-08-25T05:00:50.049 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.049 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.050 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.050 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_ssl": { 2023-08-25T05:00:50.050 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_ssl", 2023-08-25T05:00:50.050 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.051 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.052 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-25T05:00:50.052 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.052 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.052 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.053 INFO:teuthology.orchestra.run.smithi145.stdout: "smtp_user": { 2023-08-25T05:00:50.053 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "smtp_user", 2023-08-25T05:00:50.053 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.053 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.053 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.054 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.054 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.054 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.054 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.054 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "User to authenticate as", 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.056 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.056 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "balancer", 2023-08-25T05:00:50.056 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.056 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.056 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.057 INFO:teuthology.orchestra.run.smithi145.stdout: "active": { 2023-08-25T05:00:50.057 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "active", 2023-08-25T05:00:50.057 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.057 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.057 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.058 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.058 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.058 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.058 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.058 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically balance PGs across cluster", 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_time": { 2023-08-25T05:00:50.059 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_time", 2023-08-25T05:00:50.060 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.060 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.060 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.060 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0000", 2023-08-25T05:00:50.060 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.061 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.061 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.061 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "beginning time of day to automatically balance", 2023-08-25T05:00:50.061 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:50.061 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.062 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.062 INFO:teuthology.orchestra.run.smithi145.stdout: "begin_weekday": { 2023-08-25T05:00:50.062 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "begin_weekday", 2023-08-25T05:00:50.062 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:50.063 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.063 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.063 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:50.063 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:50.063 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:50.064 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.064 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-25T05:00:50.064 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:50.064 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.064 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.065 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.065 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_max_iterations": { 2023-08-25T05:00:50.065 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_max_iterations", 2023-08-25T05:00:50.065 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:50.065 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "25", 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "250", 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.066 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-25T05:00:50.067 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.067 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.067 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.067 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_metrics": { 2023-08-25T05:00:50.068 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_metrics", 2023-08-25T05:00:50.068 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.068 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.068 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.068 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "pgs,objects,bytes", 2023-08-25T05:00:50.069 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.069 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.069 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.069 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-25T05:00:50.069 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-25T05:00:50.070 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.070 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.070 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_compat_step": { 2023-08-25T05:00:50.070 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crush_compat_step", 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.5", 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0.001", 2023-08-25T05:00:50.071 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "0.999", 2023-08-25T05:00:50.072 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.072 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "aggressiveness of optimization", 2023-08-25T05:00:50.072 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-25T05:00:50.072 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.072 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.073 INFO:teuthology.orchestra.run.smithi145.stdout: "end_time": { 2023-08-25T05:00:50.073 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_time", 2023-08-25T05:00:50.073 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.073 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.074 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.074 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2359", 2023-08-25T05:00:50.074 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.074 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.074 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.075 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ending time of day to automatically balance", 2023-08-25T05:00:50.075 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-25T05:00:50.075 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.075 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.075 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "end_weekday": { 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "end_weekday", 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.076 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:50.077 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "0", 2023-08-25T05:00:50.077 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "6", 2023-08-25T05:00:50.077 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.077 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-25T05:00:50.077 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-25T05:00:50.078 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.078 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.078 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.078 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.079 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.079 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.079 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.079 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.079 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.080 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.081 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.081 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.081 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.081 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.081 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.082 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.082 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.082 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.082 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.082 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.083 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.083 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.083 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.083 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.083 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.084 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.084 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.084 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.084 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.084 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.085 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.086 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.086 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.086 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.086 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.086 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.087 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.087 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.087 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.087 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.087 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.088 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.089 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.089 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.089 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.089 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.089 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.090 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.090 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.090 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.090 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.090 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.091 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.091 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.091 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.091 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.091 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.092 INFO:teuthology.orchestra.run.smithi145.stdout: "min_score": { 2023-08-25T05:00:50.092 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_score", 2023-08-25T05:00:50.092 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.092 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.092 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-25T05:00:50.093 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.094 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.094 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.094 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.094 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:50.094 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:50.095 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.095 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.095 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.095 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "upmap", 2023-08-25T05:00:50.095 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.096 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.096 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.096 INFO:teuthology.orchestra.run.smithi145.stdout: "crush-compat", 2023-08-25T05:00:50.096 INFO:teuthology.orchestra.run.smithi145.stdout: "none", 2023-08-25T05:00:50.096 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap" 2023-08-25T05:00:50.097 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.097 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Balancer mode", 2023-08-25T05:00:50.097 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.097 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.097 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ids": { 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_ids", 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.098 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.099 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.099 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.099 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.099 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.099 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-25T05:00:50.100 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.100 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.100 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.100 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.100 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:50.101 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:50.101 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.101 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.101 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.101 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.102 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.103 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.103 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.103 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_deviation": { 2023-08-25T05:00:50.103 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_deviation", 2023-08-25T05:00:50.103 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:50.104 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.105 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.105 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-25T05:00:50.105 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-25T05:00:50.105 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.105 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.106 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.106 INFO:teuthology.orchestra.run.smithi145.stdout: "upmap_max_optimizations": { 2023-08-25T05:00:50.106 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "upmap_max_optimizations", 2023-08-25T05:00:50.106 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:50.106 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.107 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-25T05:00:50.108 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.108 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.108 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.109 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.109 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.109 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cephadm", 2023-08-25T05:00:50.109 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.109 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.110 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.110 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_down_multiplier": { 2023-08-25T05:00:50.110 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_down_multiplier", 2023-08-25T05:00:50.110 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.110 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.111 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-25T05:00:50.112 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.112 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.112 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.112 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_refresh_rate": { 2023-08-25T05:00:50.113 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_refresh_rate", 2023-08-25T05:00:50.113 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.113 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.113 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.113 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "20", 2023-08-25T05:00:50.114 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.114 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.114 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.114 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-25T05:00:50.114 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.115 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.115 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.115 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.115 INFO:teuthology.orchestra.run.smithi145.stdout: "agent_starting_port": { 2023-08-25T05:00:50.115 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "agent_starting_port", 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4721", 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.116 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.117 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.117 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-25T05:00:50.117 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.117 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.117 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.118 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.118 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_ptrace": { 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_ptrace", 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.119 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.120 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.120 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.120 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-25T05:00:50.120 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-25T05:00:50.121 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.121 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.121 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.121 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_interval": { 2023-08-25T05:00:50.121 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_interval", 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.122 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.123 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.123 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-25T05:00:50.123 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.123 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.123 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.124 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.124 INFO:teuthology.orchestra.run.smithi145.stdout: "autotune_memory_target_ratio": { 2023-08-25T05:00:50.124 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "autotune_memory_target_ratio", 2023-08-25T05:00:50.124 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.124 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.125 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.125 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0.7", 2023-08-25T05:00:50.125 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.125 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.125 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.126 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-25T05:00:50.126 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.126 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.126 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "cgroups_split": { 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cgroups_split", 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.127 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.128 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.128 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.128 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.128 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-25T05:00:50.129 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.129 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.129 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.130 INFO:teuthology.orchestra.run.smithi145.stdout: "config_checks_enabled": { 2023-08-25T05:00:50.130 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_checks_enabled", 2023-08-25T05:00:50.130 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.130 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.130 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-25T05:00:50.131 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.132 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.132 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.132 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.132 INFO:teuthology.orchestra.run.smithi145.stdout: "config_dashboard": { 2023-08-25T05:00:50.132 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_dashboard", 2023-08-25T05:00:50.133 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.133 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.133 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.133 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.133 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.134 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.134 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.134 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-25T05:00:50.134 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.134 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.135 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.135 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.135 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_alertmanager": { 2023-08-25T05:00:50.135 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_alertmanager", 2023-08-25T05:00:50.135 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.136 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.137 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:50.137 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.137 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.137 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.138 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_base": { 2023-08-25T05:00:50.138 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_base", 2023-08-25T05:00:50.138 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.138 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.138 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.139 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-25T05:00:50.139 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.139 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.139 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.139 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Container image name, without the tag", 2023-08-25T05:00:50.140 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.140 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.140 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.140 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_elasticsearch": { 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_elasticsearch", 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-25T05:00:50.141 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.142 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.142 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.142 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "elasticsearch container image", 2023-08-25T05:00:50.142 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.142 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.143 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.143 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_grafana": { 2023-08-25T05:00:50.143 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_grafana", 2023-08-25T05:00:50.143 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.144 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.144 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.144 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-25T05:00:50.144 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.144 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.145 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.145 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:50.145 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.145 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.145 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_haproxy": { 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_haproxy", 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.146 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.147 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-25T05:00:50.147 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.147 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.147 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.147 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "HAproxy container image", 2023-08-25T05:00:50.148 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.148 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.148 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.148 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_agent": { 2023-08-25T05:00:50.149 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_agent", 2023-08-25T05:00:50.149 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.149 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.149 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.149 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-25T05:00:50.150 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.150 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.150 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.150 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger agent container image", 2023-08-25T05:00:50.150 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_collector": { 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_collector", 2023-08-25T05:00:50.151 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.152 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.152 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.152 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-25T05:00:50.152 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.152 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.153 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.153 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger collector container image", 2023-08-25T05:00:50.153 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.153 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.153 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.154 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_jaeger_query": { 2023-08-25T05:00:50.154 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_jaeger_query", 2023-08-25T05:00:50.154 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.154 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.155 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Jaeger query container image", 2023-08-25T05:00:50.156 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.156 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.156 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.156 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_keepalived": { 2023-08-25T05:00:50.157 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_keepalived", 2023-08-25T05:00:50.157 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.157 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.157 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.157 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-25T05:00:50.158 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.158 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.158 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.158 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Keepalived container image", 2023-08-25T05:00:50.158 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_loki": { 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_loki", 2023-08-25T05:00:50.159 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.160 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.160 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.160 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-25T05:00:50.160 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.160 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.161 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.161 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Loki container image", 2023-08-25T05:00:50.161 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.161 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.161 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.162 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_node_exporter": { 2023-08-25T05:00:50.162 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_node_exporter", 2023-08-25T05:00:50.162 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.162 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-25T05:00:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.164 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:50.164 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.164 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.164 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.164 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_nvmeof": { 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_nvmeof", 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.165 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-08-25T05:00:50.166 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.166 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.166 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.166 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Nvme-of container image", 2023-08-25T05:00:50.166 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.167 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.167 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.167 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_prometheus": { 2023-08-25T05:00:50.167 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_prometheus", 2023-08-25T05:00:50.168 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.168 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.168 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.168 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-25T05:00:50.168 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.169 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.169 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.169 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Prometheus container image", 2023-08-25T05:00:50.169 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.169 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_promtail": { 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_promtail", 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.170 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.171 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.171 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-25T05:00:50.171 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.171 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.171 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.172 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Promtail container image", 2023-08-25T05:00:50.172 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.172 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.172 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.173 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image_snmp_gateway": { 2023-08-25T05:00:50.173 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_image_snmp_gateway", 2023-08-25T05:00:50.173 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.173 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.173 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "SNMP Gateway container image", 2023-08-25T05:00:50.174 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.175 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.175 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.175 INFO:teuthology.orchestra.run.smithi145.stdout: "container_init": { 2023-08-25T05:00:50.175 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "container_init", 2023-08-25T05:00:50.176 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.176 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.176 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.176 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.176 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.177 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.177 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.177 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Run podman/docker with `--init`", 2023-08-25T05:00:50.177 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.177 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: "daemon_cache_timeout": { 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "daemon_cache_timeout", 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.178 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.179 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.179 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:50.179 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.179 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.179 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.180 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-25T05:00:50.180 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.180 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.180 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.181 INFO:teuthology.orchestra.run.smithi145.stdout: "default_cephadm_command_timeout": { 2023-08-25T05:00:50.181 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_cephadm_command_timeout", 2023-08-25T05:00:50.181 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.181 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.181 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.182 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "900", 2023-08-25T05:00:50.182 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.182 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.182 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.182 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: "default_registry": { 2023-08-25T05:00:50.183 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "default_registry", 2023-08-25T05:00:50.184 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.184 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.184 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.184 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "docker.io", 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.185 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.186 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.186 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.186 INFO:teuthology.orchestra.run.smithi145.stdout: "device_cache_timeout": { 2023-08-25T05:00:50.186 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_cache_timeout", 2023-08-25T05:00:50.186 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.187 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.187 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.187 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1800", 2023-08-25T05:00:50.187 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.187 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.188 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.188 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache device inventory", 2023-08-25T05:00:50.188 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.188 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.188 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.189 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.189 INFO:teuthology.orchestra.run.smithi145.stdout: "device_enhanced_scan": { 2023-08-25T05:00:50.189 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_enhanced_scan", 2023-08-25T05:00:50.189 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.189 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.190 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-25T05:00:50.191 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.191 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.191 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.191 INFO:teuthology.orchestra.run.smithi145.stdout: "facts_cache_timeout": { 2023-08-25T05:00:50.192 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "facts_cache_timeout", 2023-08-25T05:00:50.192 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.192 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.192 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.192 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:50.193 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.193 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.193 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.193 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "seconds to cache host facts data", 2023-08-25T05:00:50.193 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.194 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.194 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.194 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.194 INFO:teuthology.orchestra.run.smithi145.stdout: "host_check_interval": { 2023-08-25T05:00:50.194 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "host_check_interval", 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.195 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.196 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.196 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to perform a host check", 2023-08-25T05:00:50.196 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.196 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.196 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.197 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.197 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.197 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.197 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.197 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.198 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.198 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.198 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.198 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.198 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.199 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.200 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.200 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.200 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.200 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.200 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.201 INFO:teuthology.orchestra.run.smithi145.stdout: "log_refresh_metadata": { 2023-08-25T05:00:50.201 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_refresh_metadata", 2023-08-25T05:00:50.201 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.201 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.202 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.202 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.202 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.202 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.202 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.203 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-25T05:00:50.203 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.203 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.203 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.203 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.204 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.204 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.204 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.204 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.204 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-25T05:00:50.205 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.206 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.206 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.206 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.206 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.206 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.207 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.207 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.207 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.207 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.207 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.208 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.208 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.208 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.208 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.208 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.209 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.210 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.210 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.210 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.210 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.210 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.211 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.211 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.211 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.211 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.211 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.212 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.212 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.212 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.212 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.212 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.213 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.213 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-25T05:00:50.213 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-25T05:00:50.213 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.214 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.215 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-25T05:00:50.215 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.215 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.215 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.215 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.216 INFO:teuthology.orchestra.run.smithi145.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-25T05:00:50.216 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-25T05:00:50.216 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.216 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.217 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.217 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "*", 2023-08-25T05:00:50.217 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.217 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.217 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.218 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-25T05:00:50.218 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.218 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.219 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.219 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.219 INFO:teuthology.orchestra.run.smithi145.stdout: "max_count_per_host": { 2023-08-25T05:00:50.219 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_count_per_host", 2023-08-25T05:00:50.219 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.220 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.220 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.220 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.220 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.220 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.221 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.221 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of daemons per service per host", 2023-08-25T05:00:50.221 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.221 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.222 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.222 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.222 INFO:teuthology.orchestra.run.smithi145.stdout: "max_osd_draining_count": { 2023-08-25T05:00:50.223 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_osd_draining_count", 2023-08-25T05:00:50.223 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.223 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.223 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.223 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.224 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.224 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.224 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.224 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-25T05:00:50.224 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.225 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.225 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.225 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.225 INFO:teuthology.orchestra.run.smithi145.stdout: "migration_current": { 2023-08-25T05:00:50.225 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "migration_current", 2023-08-25T05:00:50.226 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.226 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.226 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.226 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.226 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.227 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.227 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.227 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "internal - do not modify", 2023-08-25T05:00:50.227 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.227 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.228 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.228 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.228 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": { 2023-08-25T05:00:50.228 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mode", 2023-08-25T05:00:50.228 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.229 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.229 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.229 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "root", 2023-08-25T05:00:50.229 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.230 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.230 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.230 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm-package", 2023-08-25T05:00:50.230 INFO:teuthology.orchestra.run.smithi145.stdout: "root" 2023-08-25T05:00:50.230 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.231 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "mode for remote execution of cephadm", 2023-08-25T05:00:50.231 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.231 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.231 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.231 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.232 INFO:teuthology.orchestra.run.smithi145.stdout: "prometheus_alerts_path": { 2023-08-25T05:00:50.232 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus_alerts_path", 2023-08-25T05:00:50.232 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.232 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.232 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.233 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-25T05:00:50.233 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.233 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.233 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.233 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-25T05:00:50.234 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.234 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.234 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.234 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.234 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_insecure": { 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_insecure", 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.235 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.236 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.236 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.236 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-25T05:00:50.236 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.236 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.237 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.237 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.237 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_password": { 2023-08-25T05:00:50.237 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_password", 2023-08-25T05:00:50.237 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.238 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.238 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.238 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.238 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.238 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.239 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.239 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-25T05:00:50.239 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.239 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.239 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.240 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.240 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_url": { 2023-08-25T05:00:50.240 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_url", 2023-08-25T05:00:50.240 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.240 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.241 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-25T05:00:50.242 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.242 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.242 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.242 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.242 INFO:teuthology.orchestra.run.smithi145.stdout: "registry_username": { 2023-08-25T05:00:50.243 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "registry_username", 2023-08-25T05:00:50.243 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.243 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.243 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.243 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.244 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.244 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.244 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.244 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-25T05:00:50.245 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.245 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.245 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.245 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.246 INFO:teuthology.orchestra.run.smithi145.stdout: "secure_monitoring_stack": { 2023-08-25T05:00:50.246 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "secure_monitoring_stack", 2023-08-25T05:00:50.246 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.246 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.246 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.247 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.247 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.247 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.247 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.247 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-25T05:00:50.248 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.248 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.248 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.248 INFO:teuthology.orchestra.run.smithi145.stdout: "service_discovery_port": { 2023-08-25T05:00:50.249 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "service_discovery_port", 2023-08-25T05:00:50.249 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.249 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.249 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.249 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8765", 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "cephadm service discovery port", 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.250 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.251 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.252 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.252 INFO:teuthology.orchestra.run.smithi145.stdout: "ssh_config_file": { 2023-08-25T05:00:50.252 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssh_config_file", 2023-08-25T05:00:50.253 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.253 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.253 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.254 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.254 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.254 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.254 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.255 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-25T05:00:50.255 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.255 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.255 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.256 INFO:teuthology.orchestra.run.smithi145.stdout: "use_agent": { 2023-08-25T05:00:50.256 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_agent", 2023-08-25T05:00:50.257 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.257 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.257 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.257 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.257 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.258 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.258 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.258 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-25T05:00:50.258 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.259 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.259 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.259 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.259 INFO:teuthology.orchestra.run.smithi145.stdout: "use_repo_digest": { 2023-08-25T05:00:50.260 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "use_repo_digest", 2023-08-25T05:00:50.260 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.260 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.260 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.260 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.261 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.261 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.261 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.262 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-25T05:00:50.262 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.262 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.263 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.263 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.263 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_failed_host_check": { 2023-08-25T05:00:50.263 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_failed_host_check", 2023-08-25T05:00:50.264 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.264 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.264 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.264 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.265 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.265 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.265 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.265 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if the host check fails", 2023-08-25T05:00:50.266 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.266 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.266 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.266 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.266 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_daemons": { 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_daemons", 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.267 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.268 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.268 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.268 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-25T05:00:50.269 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.269 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.269 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.269 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.270 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_on_stray_hosts": { 2023-08-25T05:00:50.270 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_on_stray_hosts", 2023-08-25T05:00:50.270 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.270 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.270 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.271 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.271 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.271 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.272 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.272 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-25T05:00:50.272 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.272 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.272 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.273 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.273 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.273 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.273 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.274 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crash", 2023-08-25T05:00:50.274 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.274 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.274 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.274 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.275 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.275 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.275 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.275 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.275 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.276 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.276 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.277 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.277 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.277 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.277 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.277 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.278 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.278 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.280 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.280 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.281 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.281 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.282 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.282 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.282 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.283 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.283 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.283 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.284 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.285 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.286 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.286 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.287 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.287 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.288 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.288 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.289 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.289 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.290 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.290 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.290 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.291 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.291 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.291 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.292 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.292 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.292 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.293 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.293 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.294 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.294 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.294 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.294 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.295 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.295 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.295 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.295 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.295 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.296 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.296 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.296 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.297 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.297 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.298 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.298 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.298 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.298 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.299 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.299 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.299 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.299 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.299 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.300 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.300 INFO:teuthology.orchestra.run.smithi145.stdout: "retain_interval": { 2023-08-25T05:00:50.300 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retain_interval", 2023-08-25T05:00:50.301 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.301 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.301 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.301 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "31536000", 2023-08-25T05:00:50.302 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.302 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.302 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.302 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-25T05:00:50.302 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.303 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.303 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.303 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.303 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_recent_interval": { 2023-08-25T05:00:50.304 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_recent_interval", 2023-08-25T05:00:50.304 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.304 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.304 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.305 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1209600", 2023-08-25T05:00:50.305 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.305 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.305 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.306 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-25T05:00:50.306 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.306 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.306 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dashboard", 2023-08-25T05:00:50.307 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.308 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.308 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.309 INFO:teuthology.orchestra.run.smithi145.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-25T05:00:50.309 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-25T05:00:50.310 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.310 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.310 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.310 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.311 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.311 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.311 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.311 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.312 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.312 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.312 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.312 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.312 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_HOST": { 2023-08-25T05:00:50.313 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-25T05:00:50.313 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.314 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.314 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.314 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.315 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.315 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.315 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.316 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.317 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.317 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.317 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.318 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.318 INFO:teuthology.orchestra.run.smithi145.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-25T05:00:50.319 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-25T05:00:50.319 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.319 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.319 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.320 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.320 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.320 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.321 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.321 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.321 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.321 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.321 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.322 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.322 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_ENABLED": { 2023-08-25T05:00:50.322 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_ENABLED", 2023-08-25T05:00:50.322 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.322 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.323 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.323 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.326 INFO:teuthology.orchestra.run.smithi145.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-25T05:00:50.326 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-25T05:00:50.326 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.327 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.327 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.327 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.327 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.328 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.328 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.328 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.328 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.329 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.329 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.329 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "ENABLE_BROWSABLE_API": { 2023-08-25T05:00:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-25T05:00:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.332 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.332 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.333 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.333 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.334 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.334 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.334 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.335 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.335 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.336 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-25T05:00:50.336 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-25T05:00:50.337 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.337 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.338 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.338 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.339 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.339 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.339 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.340 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.340 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.341 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.341 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.342 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.342 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-25T05:00:50.343 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-25T05:00:50.343 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.344 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.344 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.345 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.345 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.345 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.346 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.346 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.347 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.347 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.348 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.348 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.349 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-25T05:00:50.349 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-25T05:00:50.349 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.350 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.350 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.350 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.351 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.351 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.351 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.351 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.352 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.352 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.352 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.353 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.353 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-25T05:00:50.353 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-25T05:00:50.353 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.354 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.354 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.354 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.355 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.355 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.355 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.355 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.356 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.356 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.356 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.357 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.357 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-25T05:00:50.357 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-25T05:00:50.358 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.358 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.358 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.358 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.359 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.359 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.359 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.360 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.360 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.360 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.361 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.361 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.362 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-25T05:00:50.362 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-25T05:00:50.363 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.363 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.363 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.364 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.364 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.365 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.365 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.366 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.366 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.366 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.367 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.367 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.368 INFO:teuthology.orchestra.run.smithi145.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-25T05:00:50.368 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-25T05:00:50.368 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.369 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.369 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.370 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.370 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.371 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.371 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.371 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.372 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.372 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.373 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.373 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.374 INFO:teuthology.orchestra.run.smithi145.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-25T05:00:50.374 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-25T05:00:50.374 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.375 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.375 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.376 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.376 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.376 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.377 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.377 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.378 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.378 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.378 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.379 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.379 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_PASSWORD": { 2023-08-25T05:00:50.380 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-25T05:00:50.380 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.380 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.381 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.381 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:50.381 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.382 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.382 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.382 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.383 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.383 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.383 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.383 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.384 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-25T05:00:50.384 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-25T05:00:50.384 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.385 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.385 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.385 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.385 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.386 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.386 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.386 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.387 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.387 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.387 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.387 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.387 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_URL": { 2023-08-25T05:00:50.388 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_URL", 2023-08-25T05:00:50.388 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.388 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.389 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.389 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.389 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.389 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.390 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.390 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.390 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.391 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.391 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.391 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.391 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_API_USERNAME": { 2023-08-25T05:00:50.392 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-25T05:00:50.392 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.393 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.393 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.393 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:50.394 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.394 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.394 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.394 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.395 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.395 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.396 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.396 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.397 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-25T05:00:50.397 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-25T05:00:50.398 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.398 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.398 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.399 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.399 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.400 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.400 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.400 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.401 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.401 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.401 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.401 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.401 INFO:teuthology.orchestra.run.smithi145.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-25T05:00:50.402 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-25T05:00:50.402 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.402 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.402 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.403 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.403 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.403 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.404 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.404 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.404 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.404 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.405 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.406 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.406 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.406 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.407 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.407 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.407 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.407 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.407 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.408 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.408 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.408 INFO:teuthology.orchestra.run.smithi145.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-25T05:00:50.408 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-25T05:00:50.408 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.409 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.409 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.409 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.409 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.409 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.410 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_HOST": { 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.411 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.412 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.412 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.412 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.413 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.413 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.413 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.414 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.414 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.414 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.415 INFO:teuthology.orchestra.run.smithi145.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-25T05:00:50.415 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-25T05:00:50.415 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.415 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.416 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.416 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.417 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.417 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.417 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.418 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.418 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.418 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-25T05:00:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-25T05:00:50.419 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.420 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.420 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.420 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.420 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.421 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.421 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.421 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.421 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.421 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.422 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.423 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.423 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.423 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.423 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.423 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.424 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.424 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.424 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.424 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.424 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.425 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-25T05:00:50.425 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-25T05:00:50.425 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.425 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.425 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.426 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.426 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.426 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.426 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.426 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.427 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.427 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.427 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.427 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.427 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-25T05:00:50.428 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-25T05:00:50.428 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.428 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.428 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.428 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.429 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.429 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.429 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.429 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.429 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.430 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.430 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.430 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.430 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-25T05:00:50.430 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-25T05:00:50.431 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.431 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.431 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.431 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.432 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.432 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.432 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.432 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.432 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.433 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.433 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.433 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.433 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-25T05:00:50.433 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-25T05:00:50.434 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.434 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.434 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.434 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.434 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.435 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.435 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.435 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.435 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.435 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.436 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.436 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.436 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-25T05:00:50.436 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-25T05:00:50.436 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.437 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.437 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.437 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.437 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.437 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.438 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.438 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.438 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.438 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.438 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.439 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.439 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_ENABLED": { 2023-08-25T05:00:50.439 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-25T05:00:50.439 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.439 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.440 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.440 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.440 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.440 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.440 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.441 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.441 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.441 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.441 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.442 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.442 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-25T05:00:50.442 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-25T05:00:50.442 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.443 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.443 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.443 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-25T05:00:50.443 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.444 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.444 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.444 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.444 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.444 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.445 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.445 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.445 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-25T05:00:50.445 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-25T05:00:50.445 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.446 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.446 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.446 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.446 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.446 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.447 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.447 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.447 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.447 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.447 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.448 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.448 INFO:teuthology.orchestra.run.smithi145.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-25T05:00:50.448 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-25T05:00:50.448 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8", 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.449 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.450 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.450 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.450 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.450 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.450 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.451 INFO:teuthology.orchestra.run.smithi145.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-25T05:00:50.451 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-25T05:00:50.451 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.451 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.451 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.452 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "45", 2023-08-25T05:00:50.452 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.452 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.452 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.452 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.453 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.453 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.453 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.453 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.453 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ACCESS_KEY": { 2023-08-25T05:00:50.454 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-25T05:00:50.454 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.454 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.454 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.454 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.455 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.455 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.455 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.455 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.455 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.456 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.456 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.456 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.456 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-25T05:00:50.456 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-25T05:00:50.457 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.457 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.457 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.457 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "admin", 2023-08-25T05:00:50.457 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.458 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.458 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.458 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.458 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.458 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.459 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.459 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.459 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SECRET_KEY": { 2023-08-25T05:00:50.459 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-25T05:00:50.459 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.460 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.460 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.460 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.460 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.460 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.461 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.461 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.461 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.461 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.461 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: "RGW_API_SSL_VERIFY": { 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.462 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.463 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.463 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.463 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.463 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.463 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.464 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.464 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.464 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.464 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.464 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-25T05:00:50.465 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-25T05:00:50.465 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.465 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.465 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.466 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:50.466 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.466 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.466 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.466 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.467 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.467 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.467 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.467 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.467 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-25T05:00:50.468 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-25T05:00:50.468 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.468 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.468 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.468 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.469 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.469 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.469 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.469 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.469 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.470 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.470 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.470 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.470 INFO:teuthology.orchestra.run.smithi145.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-25T05:00:50.470 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-25T05:00:50.471 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.471 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.471 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.471 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:50.471 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.472 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.472 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.472 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.472 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.472 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.473 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.473 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.473 INFO:teuthology.orchestra.run.smithi145.stdout: "cross_origin_url": { 2023-08-25T05:00:50.473 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cross_origin_url", 2023-08-25T05:00:50.473 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.474 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.475 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.475 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.475 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.475 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.475 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.476 INFO:teuthology.orchestra.run.smithi145.stdout: "crt_file": { 2023-08-25T05:00:50.476 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "crt_file", 2023-08-25T05:00:50.476 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.476 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.476 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.477 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.477 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.477 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.477 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.477 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.478 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.478 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.478 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.478 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.478 INFO:teuthology.orchestra.run.smithi145.stdout: "debug": { 2023-08-25T05:00:50.479 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "debug", 2023-08-25T05:00:50.479 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.479 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.479 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.479 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.480 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.480 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.480 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.480 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Enable/disable debug options", 2023-08-25T05:00:50.480 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.481 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.481 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.481 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.481 INFO:teuthology.orchestra.run.smithi145.stdout: "jwt_token_ttl": { 2023-08-25T05:00:50.481 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "jwt_token_ttl", 2023-08-25T05:00:50.482 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.482 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.482 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.482 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "28800", 2023-08-25T05:00:50.482 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.483 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.483 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.483 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.483 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.483 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.484 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.484 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.484 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:50.484 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:50.484 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.485 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.485 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.485 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.485 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.485 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.486 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.487 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.487 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.487 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.487 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.487 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.488 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.488 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.488 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.488 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.488 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.489 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.489 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.489 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.489 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.489 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.490 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.490 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.490 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.490 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.490 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.491 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.492 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.492 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.492 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.492 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.492 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.493 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.493 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.493 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.493 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.493 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.494 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.494 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.494 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.494 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.494 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.496 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.496 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.496 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.496 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.496 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.497 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.497 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.497 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.497 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.497 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.498 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.498 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.498 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.498 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.498 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.499 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.499 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.499 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.499 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.499 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.500 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.500 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.500 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.500 INFO:teuthology.orchestra.run.smithi145.stdout: "motd": { 2023-08-25T05:00:50.500 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "motd", 2023-08-25T05:00:50.501 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.501 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.501 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.501 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.501 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.502 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.502 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.502 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "The message of the day", 2023-08-25T05:00:50.502 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.502 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.503 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.503 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.503 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect_resolve_ip_addr": { 2023-08-25T05:00:50.503 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "redirect_resolve_ip_addr", 2023-08-25T05:00:50.503 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.504 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.505 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.505 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.505 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.505 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.505 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.506 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:50.506 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:50.506 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.506 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.506 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.507 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:50.507 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.507 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.507 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.507 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.508 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.508 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.508 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.508 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.508 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:50.509 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:50.509 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.509 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.509 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.509 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8080", 2023-08-25T05:00:50.510 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.510 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.510 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.510 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.510 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:50.511 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.512 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.512 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.512 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.512 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.512 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.513 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.513 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.513 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.513 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.513 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.514 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.514 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl_server_port": { 2023-08-25T05:00:50.514 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl_server_port", 2023-08-25T05:00:50.514 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.514 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.515 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.515 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8443", 2023-08-25T05:00:50.515 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.515 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.515 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.516 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.516 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.516 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.516 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.516 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.517 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "redirect", 2023-08-25T05:00:50.518 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.518 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.518 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.518 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.518 INFO:teuthology.orchestra.run.smithi145.stdout: "redirect" 2023-08-25T05:00:50.519 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.519 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.519 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.519 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.520 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.520 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.520 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:50.520 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:50.520 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.521 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.521 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.521 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:50.521 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:50.521 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:50.522 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.522 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.522 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.522 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.522 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.523 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.523 INFO:teuthology.orchestra.run.smithi145.stdout: "url_prefix": { 2023-08-25T05:00:50.523 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url_prefix", 2023-08-25T05:00:50.523 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.523 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.524 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.524 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.524 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.524 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.524 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.525 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "devicehealth", 2023-08-25T05:00:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.527 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.527 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_monitoring": { 2023-08-25T05:00:50.527 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_monitoring", 2023-08-25T05:00:50.527 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.527 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.528 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.528 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.528 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.528 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.528 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "monitor device health metrics", 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.529 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.530 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.530 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.530 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.530 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.530 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.531 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.531 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.531 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.531 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.531 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.532 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.532 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.532 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.532 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.532 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.533 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.533 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.533 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.533 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.533 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.534 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.534 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.534 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.534 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.534 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.535 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.535 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.535 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.535 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.535 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.536 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.536 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.536 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.536 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.536 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.537 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.537 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.537 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.537 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.537 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.538 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.539 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.539 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.539 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.539 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.539 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.540 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.540 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.540 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.540 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.540 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.541 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.541 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.541 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.541 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.541 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.542 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.542 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.542 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.542 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.542 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.543 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.543 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.543 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.543 INFO:teuthology.orchestra.run.smithi145.stdout: "mark_out_threshold": { 2023-08-25T05:00:50.543 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mark_out_threshold", 2023-08-25T05:00:50.544 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.544 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.544 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.544 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "2419200", 2023-08-25T05:00:50.544 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.545 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.545 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.545 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-25T05:00:50.545 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.545 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.546 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.546 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.546 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_name": { 2023-08-25T05:00:50.546 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pool_name", 2023-08-25T05:00:50.546 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.547 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.547 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.547 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "device_health_metrics", 2023-08-25T05:00:50.547 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.547 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.548 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.548 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-25T05:00:50.548 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.548 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.549 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.549 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.549 INFO:teuthology.orchestra.run.smithi145.stdout: "retention_period": { 2023-08-25T05:00:50.549 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "retention_period", 2023-08-25T05:00:50.549 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.550 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.550 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.550 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15552000", 2023-08-25T05:00:50.550 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.550 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long to retain device health metrics", 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.551 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.552 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_frequency": { 2023-08-25T05:00:50.552 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_frequency", 2023-08-25T05:00:50.552 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.552 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.553 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-25T05:00:50.554 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.554 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.554 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.554 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.554 INFO:teuthology.orchestra.run.smithi145.stdout: "self_heal": { 2023-08-25T05:00:50.555 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "self_heal", 2023-08-25T05:00:50.555 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.555 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.555 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.555 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.556 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.556 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.556 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.556 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-25T05:00:50.556 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.557 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.557 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.557 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.557 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:50.557 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:50.558 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.558 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.558 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.558 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:50.558 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.559 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.559 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.559 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how frequently to wake up and check device health", 2023-08-25T05:00:50.559 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.559 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.560 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.560 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.560 INFO:teuthology.orchestra.run.smithi145.stdout: "warn_threshold": { 2023-08-25T05:00:50.560 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "warn_threshold", 2023-08-25T05:00:50.560 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.561 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.561 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.561 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7257600", 2023-08-25T05:00:50.561 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.561 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.562 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.563 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.563 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.563 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.563 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "diskprediction_local", 2023-08-25T05:00:50.563 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.564 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.564 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.564 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.564 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.564 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.565 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.565 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.565 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.565 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.566 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.567 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.567 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.567 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.567 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.567 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.568 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.568 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.568 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.568 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.568 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.569 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.569 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.569 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.569 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.569 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.570 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.570 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.570 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.570 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.570 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.571 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.571 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.571 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.571 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.571 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.572 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.572 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.572 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.572 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.572 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.573 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.573 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.573 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.573 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.573 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.574 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.574 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.574 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.574 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.574 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.575 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.575 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.575 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.575 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.575 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.576 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.576 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.576 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.576 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.576 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.577 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.577 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.577 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.577 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.577 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: "predict_interval": { 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predict_interval", 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.578 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.579 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.579 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "86400", 2023-08-25T05:00:50.579 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.579 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.579 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.580 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.580 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.580 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.580 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.580 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.581 INFO:teuthology.orchestra.run.smithi145.stdout: "predictor_model": { 2023-08-25T05:00:50.581 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "predictor_model", 2023-08-25T05:00:50.581 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.581 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.581 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "prophetstor", 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.582 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.583 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.583 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.583 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.583 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:50.583 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:50.584 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.584 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.584 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.584 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "600", 2023-08-25T05:00:50.584 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.585 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.585 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.585 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.585 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.585 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.586 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.586 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.586 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.586 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.586 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.587 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "influx", 2023-08-25T05:00:50.587 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": false, 2023-08-25T05:00:50.587 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "influxdb python module not found", 2023-08-25T05:00:50.587 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.587 INFO:teuthology.orchestra.run.smithi145.stdout: "batch_size": { 2023-08-25T05:00:50.588 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "batch_size", 2023-08-25T05:00:50.588 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.588 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.588 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.588 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5000", 2023-08-25T05:00:50.589 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.589 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.589 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.589 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-25T05:00:50.589 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.590 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.590 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.590 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.590 INFO:teuthology.orchestra.run.smithi145.stdout: "database": { 2023-08-25T05:00:50.590 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "database", 2023-08-25T05:00:50.591 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.591 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.591 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.591 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "ceph", 2023-08-25T05:00:50.592 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.592 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.592 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.592 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-25T05:00:50.592 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.593 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.593 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.593 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.593 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": { 2023-08-25T05:00:50.594 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "hostname", 2023-08-25T05:00:50.594 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.594 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.594 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.594 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server hostname", 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.595 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.596 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.596 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.596 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:50.596 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "30", 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "5", 2023-08-25T05:00:50.597 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.598 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.598 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-25T05:00:50.598 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.598 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.598 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.599 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.599 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.599 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.599 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.599 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.600 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.600 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.600 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.600 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.600 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.601 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.601 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.601 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.601 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.601 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.602 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.602 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.602 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.602 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.602 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.603 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.603 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.603 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.603 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.603 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.604 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.604 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.604 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.604 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.604 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.605 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.606 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.606 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.606 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.606 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.606 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.607 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.607 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.607 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.607 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.607 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.608 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.608 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.608 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.608 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.608 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.610 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.610 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.610 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.610 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.611 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.612 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.612 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.612 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.612 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.612 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.613 INFO:teuthology.orchestra.run.smithi145.stdout: "password": { 2023-08-25T05:00:50.613 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "password", 2023-08-25T05:00:50.613 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.613 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.613 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.614 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.614 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.614 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.614 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.614 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "password of InfluxDB server user", 2023-08-25T05:00:50.615 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.615 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.615 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.615 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.615 INFO:teuthology.orchestra.run.smithi145.stdout: "port": { 2023-08-25T05:00:50.616 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "port", 2023-08-25T05:00:50.616 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.616 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.616 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.616 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "8086", 2023-08-25T05:00:50.617 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.617 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.617 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.617 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "InfluxDB server port", 2023-08-25T05:00:50.617 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.618 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.618 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.618 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.618 INFO:teuthology.orchestra.run.smithi145.stdout: "ssl": { 2023-08-25T05:00:50.618 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ssl", 2023-08-25T05:00:50.619 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.619 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.619 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.619 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "false", 2023-08-25T05:00:50.619 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.620 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.620 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.620 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:50.620 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.620 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: "threads": { 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threads", 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.621 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.622 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.622 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:50.622 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:50.622 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "32", 2023-08-25T05:00:50.622 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.623 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-25T05:00:50.623 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.623 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.623 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.623 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.624 INFO:teuthology.orchestra.run.smithi145.stdout: "username": { 2023-08-25T05:00:50.624 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "username", 2023-08-25T05:00:50.624 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.624 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.624 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.625 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.625 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.625 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.625 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.625 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "username of InfluxDB server user", 2023-08-25T05:00:50.626 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.626 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.626 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.626 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.626 INFO:teuthology.orchestra.run.smithi145.stdout: "verify_ssl": { 2023-08-25T05:00:50.627 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "verify_ssl", 2023-08-25T05:00:50.627 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.627 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.627 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.627 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "true", 2023-08-25T05:00:50.628 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.628 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.628 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.628 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-25T05:00:50.628 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.629 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.629 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.629 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.629 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.629 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.630 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.630 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "insights", 2023-08-25T05:00:50.630 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.630 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.630 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.631 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.631 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.631 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.631 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.631 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.632 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.633 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.633 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.633 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.633 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.633 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.634 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.634 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.634 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.634 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.634 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.635 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.635 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.635 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.635 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.635 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.636 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.636 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.636 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.636 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.636 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.637 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.637 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.637 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.637 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.637 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.638 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.638 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.638 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.638 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.638 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.639 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.639 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.639 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.639 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.639 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.640 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.640 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.640 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.640 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.640 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.641 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.641 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.641 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.641 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.641 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.642 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.642 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.642 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.642 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.642 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.643 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.643 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.643 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.643 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.643 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.644 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.644 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.644 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.644 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.644 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.645 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.645 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.645 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.645 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "iostat", 2023-08-25T05:00:50.645 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.646 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.646 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.646 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.646 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.646 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.647 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.647 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.647 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.647 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.648 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.648 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.648 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.648 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.648 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.649 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.649 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.649 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.649 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.649 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.650 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.650 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.650 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.650 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.650 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.651 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.651 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.651 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.651 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.651 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.652 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.652 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.652 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.652 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.653 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.653 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.653 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.653 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.653 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.654 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.655 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.655 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.655 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.655 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.655 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.656 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.656 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.656 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.656 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.656 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.657 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.657 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.657 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.657 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.657 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.658 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.658 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.658 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.658 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.658 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.659 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.660 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.660 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.660 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.660 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.660 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.661 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "k8sevents", 2023-08-25T05:00:50.661 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.661 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.661 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.661 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_event_retention_days": { 2023-08-25T05:00:50.662 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "ceph_event_retention_days", 2023-08-25T05:00:50.662 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.662 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.662 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.662 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "7", 2023-08-25T05:00:50.663 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.663 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.663 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.663 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-25T05:00:50.663 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.664 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.664 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.664 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.664 INFO:teuthology.orchestra.run.smithi145.stdout: "config_check_secs": { 2023-08-25T05:00:50.664 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "config_check_secs", 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "10", 2023-08-25T05:00:50.665 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.666 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.666 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-25T05:00:50.666 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.666 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.666 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.667 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.667 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.667 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.667 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.668 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.668 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.668 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.668 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.668 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.669 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.669 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.669 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.669 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.669 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.670 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.670 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.670 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.670 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.670 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.671 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.671 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.671 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.671 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.672 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.672 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.672 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.672 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.672 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.674 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.674 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.674 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.674 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.674 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.675 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.675 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.675 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.675 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.675 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.676 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.676 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.676 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.676 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.676 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.678 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.678 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.678 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.678 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.678 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.679 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.679 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.679 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.679 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.679 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.680 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.680 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.680 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.680 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.680 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "localpool", 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.681 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.682 INFO:teuthology.orchestra.run.smithi145.stdout: "failure_domain": { 2023-08-25T05:00:50.682 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "failure_domain", 2023-08-25T05:00:50.682 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.682 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.682 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.683 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "host", 2023-08-25T05:00:50.683 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.683 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.683 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.683 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "failure domain for any created local pool", 2023-08-25T05:00:50.684 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-25T05:00:50.684 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.684 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.684 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.685 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.685 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.685 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.685 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.685 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.686 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.687 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.687 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.687 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.687 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.687 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.688 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.688 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.688 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.688 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.688 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.689 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.689 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.689 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.689 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.689 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.690 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.690 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.690 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.690 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.690 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.691 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.692 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.692 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.692 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.692 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.693 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.693 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.693 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.693 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.693 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.694 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.694 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.694 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.694 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.694 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.695 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.695 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.695 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.695 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.695 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.696 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.697 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.697 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.697 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.697 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.697 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.698 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.698 INFO:teuthology.orchestra.run.smithi145.stdout: "min_size": { 2023-08-25T05:00:50.698 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "min_size", 2023-08-25T05:00:50.698 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.699 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.699 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.699 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.699 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.699 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.700 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.700 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default min_size for any created local pool", 2023-08-25T05:00:50.700 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-25T05:00:50.700 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.700 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.701 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.701 INFO:teuthology.orchestra.run.smithi145.stdout: "num_rep": { 2023-08-25T05:00:50.701 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "num_rep", 2023-08-25T05:00:50.701 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.701 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.702 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.702 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3", 2023-08-25T05:00:50.702 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.702 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.702 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.703 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default replica count for any created local pool", 2023-08-25T05:00:50.703 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.703 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.703 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.704 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_num": { 2023-08-25T05:00:50.704 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_num", 2023-08-25T05:00:50.704 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.704 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.704 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.705 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "128", 2023-08-25T05:00:50.705 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.705 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.705 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.705 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "default pg_num for any created local pool", 2023-08-25T05:00:50.706 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.706 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.706 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.706 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.706 INFO:teuthology.orchestra.run.smithi145.stdout: "prefix": { 2023-08-25T05:00:50.707 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prefix", 2023-08-25T05:00:50.707 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.707 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.707 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.707 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.708 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.708 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.708 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.708 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "name prefix for any created local pool", 2023-08-25T05:00:50.708 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: "subtree": { 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "subtree", 2023-08-25T05:00:50.709 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.710 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.710 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.710 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "rack", 2023-08-25T05:00:50.710 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.710 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.711 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.711 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-25T05:00:50.711 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-25T05:00:50.711 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.711 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.712 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.712 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.712 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.712 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.712 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mds_autoscaler", 2023-08-25T05:00:50.713 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.713 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.713 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.713 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.713 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.714 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.714 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.714 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.714 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.714 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.715 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.715 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.715 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.715 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.715 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.716 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.716 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.716 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.716 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.717 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.717 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.717 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.717 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.717 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.718 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.718 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.718 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.718 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.718 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.719 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.719 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.719 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.719 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.719 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.720 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.720 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.720 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.720 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.720 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.721 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.721 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.721 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.721 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.721 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.722 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.722 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.722 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.722 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.722 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.723 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.723 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.723 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.723 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.723 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.724 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.724 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.724 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.724 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.724 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.725 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.725 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.725 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.725 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.725 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.726 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.727 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.727 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.727 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.727 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.728 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.728 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirroring", 2023-08-25T05:00:50.728 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.728 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.728 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.729 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.729 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.729 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.729 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.729 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.730 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.730 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.730 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.730 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.730 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.731 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.731 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.731 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.731 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.731 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.732 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.732 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.732 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.732 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.732 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.733 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.733 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.733 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.733 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.734 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.735 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.735 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.735 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.735 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.735 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.736 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.736 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.736 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.736 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.736 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.737 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.737 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.737 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.737 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.737 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.738 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.738 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.738 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.738 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.738 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.739 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.739 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.739 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.739 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.739 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.740 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.740 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.740 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.740 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.740 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.741 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.741 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.741 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.741 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.741 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.742 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.742 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.742 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.742 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.742 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.743 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.743 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "nfs", 2023-08-25T05:00:50.743 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.743 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.744 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.744 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.744 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.744 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.744 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.745 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.745 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.745 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.745 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.745 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.747 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.747 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.747 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.747 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.747 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.748 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.748 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.748 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.748 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.748 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.749 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.749 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.749 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.749 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.749 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.750 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.751 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.751 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.751 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.751 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.751 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.752 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.752 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.752 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.752 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.752 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.753 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.753 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.753 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.753 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.754 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.755 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.755 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.755 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.755 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.756 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.756 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.756 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.756 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.756 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.757 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.757 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.757 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.757 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.757 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.758 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.758 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.758 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.758 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:50.758 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "fail_fs": { 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "fail_fs", 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.759 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.760 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.760 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.760 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.760 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.760 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.761 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-25T05:00:50.761 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.761 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.761 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.762 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.762 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.762 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.762 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.762 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.764 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.764 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.764 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.764 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.764 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.766 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.766 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.766 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.766 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.766 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.767 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.767 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.767 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.767 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.767 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.768 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.768 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.768 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.768 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.768 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.769 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.769 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.769 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.769 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.770 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.772 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.772 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.772 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.772 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.773 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.773 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.773 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.773 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.773 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.774 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.774 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.774 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.774 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.774 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.775 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.775 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.775 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.775 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.775 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator": { 2023-08-25T05:00:50.776 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "orchestrator", 2023-08-25T05:00:50.776 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.776 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.776 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.777 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.777 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.777 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.777 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.777 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm", 2023-08-25T05:00:50.778 INFO:teuthology.orchestra.run.smithi145.stdout: "rook", 2023-08-25T05:00:50.778 INFO:teuthology.orchestra.run.smithi145.stdout: "test_orchestrator" 2023-08-25T05:00:50.778 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.778 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Orchestrator backend", 2023-08-25T05:00:50.778 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.779 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_perf_query", 2023-08-25T05:00:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.780 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.781 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.782 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.782 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.782 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.782 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.782 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.783 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.783 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.783 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.783 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.783 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.784 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.784 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.784 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.784 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.785 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.785 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.785 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.785 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.785 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.786 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.786 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.786 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.786 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.786 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.787 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.788 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.788 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.788 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.788 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.789 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.790 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.791 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.791 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.791 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.791 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.792 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.792 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.792 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.792 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.792 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.793 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.793 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.793 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.793 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.793 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.794 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.794 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.794 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.794 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.794 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.795 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.795 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "osd_support", 2023-08-25T05:00:50.795 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.795 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.796 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.796 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.796 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.796 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.796 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.797 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.797 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.797 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.797 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.797 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.798 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.799 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.799 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.799 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.799 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.800 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.800 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.800 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.800 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.801 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.802 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.802 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.802 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.802 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.803 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.803 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.803 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.803 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.803 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.804 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.804 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.804 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.804 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.804 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.805 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.805 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.805 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.805 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.805 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.806 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.806 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.806 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.806 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.806 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.807 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.807 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.807 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.807 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.808 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.808 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.808 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.808 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.808 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.809 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.810 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.810 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.810 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "pg_autoscaler", 2023-08-25T05:00:50.810 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.810 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.811 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.811 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.811 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.811 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.811 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.812 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.812 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.812 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.812 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.812 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.813 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.813 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.813 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.813 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.813 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.814 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.814 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.814 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.814 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.814 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.815 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.816 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.816 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.816 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.816 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.816 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.817 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.817 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.817 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.817 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.817 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.818 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.818 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.818 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.818 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.818 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.819 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.819 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.819 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.819 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.819 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.820 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.820 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.820 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.820 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.820 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.821 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.821 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.821 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.821 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.821 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.822 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.823 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.823 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.823 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.823 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.823 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.824 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.824 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.824 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.824 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.824 INFO:teuthology.orchestra.run.smithi145.stdout: "noautoscale": { 2023-08-25T05:00:50.825 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "noautoscale", 2023-08-25T05:00:50.825 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.825 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.825 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.825 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.826 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.826 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.826 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.826 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "global autoscale flag", 2023-08-25T05:00:50.827 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-25T05:00:50.827 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.827 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.827 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.827 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:50.828 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:50.828 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.828 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.828 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.828 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:50.829 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.829 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.829 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.829 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.829 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.830 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.830 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.830 INFO:teuthology.orchestra.run.smithi145.stdout: "threshold": { 2023-08-25T05:00:50.830 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "threshold", 2023-08-25T05:00:50.831 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.831 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.831 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.831 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "3.0", 2023-08-25T05:00:50.831 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1.0", 2023-08-25T05:00:50.832 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.832 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.832 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "scaling threshold", 2023-08-25T05:00:50.832 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-25T05:00:50.832 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.833 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.833 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.833 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.833 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.833 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "progress", 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_pg_recovery_event": { 2023-08-25T05:00:50.834 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_pg_recovery_event", 2023-08-25T05:00:50.835 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.835 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.835 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.835 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.836 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.836 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.836 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.836 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-25T05:00:50.836 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:50.837 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.838 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.838 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.838 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.838 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.838 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.839 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.839 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.839 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.839 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.839 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.840 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.840 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.840 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.840 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.840 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.841 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.841 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.841 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.841 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.841 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.842 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.842 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.842 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.842 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.842 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.843 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.844 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.844 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.844 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.844 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.845 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.846 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.847 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.847 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.847 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.847 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.847 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.848 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.848 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.848 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.848 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.848 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.849 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.849 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.849 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.849 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.849 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.850 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.850 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.850 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.850 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.850 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.851 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.851 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.851 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.851 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.851 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.852 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.852 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.852 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.852 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.852 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.853 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.853 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.853 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.853 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.853 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.854 INFO:teuthology.orchestra.run.smithi145.stdout: "max_completed_events": { 2023-08-25T05:00:50.854 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_completed_events", 2023-08-25T05:00:50.854 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.854 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.854 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.855 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "50", 2023-08-25T05:00:50.855 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.855 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.855 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.855 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "number of past completed events to remember", 2023-08-25T05:00:50.856 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.856 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.856 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.856 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.856 INFO:teuthology.orchestra.run.smithi145.stdout: "sleep_interval": { 2023-08-25T05:00:50.857 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "sleep_interval", 2023-08-25T05:00:50.857 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:50.857 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.857 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.857 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "5", 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "how long the module is going to sleep", 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.858 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.859 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.859 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.859 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.859 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.859 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.860 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "prometheus", 2023-08-25T05:00:50.860 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.860 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.860 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.860 INFO:teuthology.orchestra.run.smithi145.stdout: "cache": { 2023-08-25T05:00:50.861 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "cache", 2023-08-25T05:00:50.861 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.861 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.861 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.861 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.862 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.862 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.862 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.862 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.862 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.863 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.863 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.863 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.863 INFO:teuthology.orchestra.run.smithi145.stdout: "exclude_perf_counters": { 2023-08-25T05:00:50.863 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "exclude_perf_counters", 2023-08-25T05:00:50.864 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.864 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.864 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.864 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.864 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.865 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.866 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.866 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.866 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.866 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.866 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.871 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.872 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.872 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.872 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.872 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.872 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.873 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.873 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.873 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.873 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.873 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.874 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.874 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.874 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.874 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.874 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.875 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.875 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.875 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.875 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.875 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.876 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.877 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.877 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.877 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.877 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.877 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.878 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.878 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.878 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.878 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.878 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.879 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.879 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.879 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.879 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.879 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools": { 2023-08-25T05:00:50.880 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools", 2023-08-25T05:00:50.880 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.880 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.880 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.880 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.881 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.881 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.881 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.881 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.881 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.882 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.882 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.882 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.882 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-25T05:00:50.882 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-25T05:00:50.883 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.883 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.883 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.883 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300", 2023-08-25T05:00:50.883 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.884 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.884 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.884 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.884 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.884 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: "scrape_interval": { 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "scrape_interval", 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.885 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.886 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.886 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15.0", 2023-08-25T05:00:50.886 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.886 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.886 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.887 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.887 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.887 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.887 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.888 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:50.888 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:50.888 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.888 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.888 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.889 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "::", 2023-08-25T05:00:50.889 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.889 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.889 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.889 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-25T05:00:50.890 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.890 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.890 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.890 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.890 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "9283", 2023-08-25T05:00:50.891 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-25T05:00:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.892 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.893 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.893 INFO:teuthology.orchestra.run.smithi145.stdout: "stale_cache_strategy": { 2023-08-25T05:00:50.893 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stale_cache_strategy", 2023-08-25T05:00:50.893 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.894 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.894 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.894 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "log", 2023-08-25T05:00:50.894 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.894 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.895 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.895 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.895 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.895 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.895 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.896 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.896 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_behaviour": { 2023-08-25T05:00:50.896 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_behaviour", 2023-08-25T05:00:50.896 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.896 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.897 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.897 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "default", 2023-08-25T05:00:50.897 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.897 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.897 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: "default", 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: "error" 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.898 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.899 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.899 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.899 INFO:teuthology.orchestra.run.smithi145.stdout: "standby_error_status_code": { 2023-08-25T05:00:50.899 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "standby_error_status_code", 2023-08-25T05:00:50.899 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.900 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.900 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.900 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "500", 2023-08-25T05:00:50.900 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "400", 2023-08-25T05:00:50.900 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "599", 2023-08-25T05:00:50.901 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.901 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.901 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.901 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.901 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.902 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.902 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.902 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.902 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.902 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_support", 2023-08-25T05:00:50.903 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.903 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.903 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.903 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.903 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.904 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.904 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.904 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.904 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.904 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.905 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.905 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.905 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.905 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.905 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.906 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.907 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.907 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.907 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.907 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.907 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.908 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.908 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.908 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.908 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.908 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.909 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.909 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.909 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.909 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.909 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.910 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.910 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.910 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.910 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.910 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.911 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.912 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.912 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.912 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.912 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.912 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.913 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.913 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.913 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.913 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.913 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.914 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.914 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.914 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.914 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.914 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.915 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.915 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.915 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.915 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.915 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.917 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.917 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_snap_create": { 2023-08-25T05:00:50.917 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_snap_create", 2023-08-25T05:00:50.917 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:50.917 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.918 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.918 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10", 2023-08-25T05:00:50.918 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.918 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.918 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.919 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.919 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.919 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.919 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.919 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.920 INFO:teuthology.orchestra.run.smithi145.stdout: "mirror_snapshot_schedule": { 2023-08-25T05:00:50.920 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mirror_snapshot_schedule", 2023-08-25T05:00:50.920 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.920 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.920 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.921 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.922 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.922 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.922 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.922 INFO:teuthology.orchestra.run.smithi145.stdout: "trash_purge_schedule": { 2023-08-25T05:00:50.922 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "trash_purge_schedule", 2023-08-25T05:00:50.923 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.923 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.923 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.923 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.923 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.924 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.924 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.924 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.924 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.924 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.925 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.925 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.925 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.925 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.925 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.926 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "restful", 2023-08-25T05:00:50.926 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.926 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.926 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.926 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_auth": { 2023-08-25T05:00:50.927 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enable_auth", 2023-08-25T05:00:50.927 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.927 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.927 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.927 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:50.928 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.928 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.928 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.928 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.928 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.929 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.929 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.929 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.929 INFO:teuthology.orchestra.run.smithi145.stdout: "key_file": { 2023-08-25T05:00:50.929 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "key_file", 2023-08-25T05:00:50.930 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.930 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.930 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.930 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.930 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.931 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.932 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.932 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.932 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.932 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.932 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.933 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.933 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.933 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.933 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.933 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.934 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.934 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.934 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.934 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.934 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.935 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.935 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.935 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.935 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.935 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.936 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.936 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.936 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.936 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.936 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.937 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.937 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.937 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.937 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.937 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.938 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.938 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.938 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.938 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.938 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.939 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.940 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.940 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.940 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.940 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.940 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.941 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.941 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.941 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.941 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.941 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.942 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.942 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.942 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.942 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.942 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.943 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.943 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.943 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.944 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.944 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.944 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.944 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.944 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.945 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.945 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.945 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.945 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.946 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.946 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.946 INFO:teuthology.orchestra.run.smithi145.stdout: "server_addr": { 2023-08-25T05:00:50.946 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_addr", 2023-08-25T05:00:50.946 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.947 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.947 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.947 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.947 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.948 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.948 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.948 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.949 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.949 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.949 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.949 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.950 INFO:teuthology.orchestra.run.smithi145.stdout: "server_port": { 2023-08-25T05:00:50.950 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "server_port", 2023-08-25T05:00:50.950 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.950 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.951 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.951 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.951 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.951 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.952 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.952 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.952 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.953 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.953 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.953 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.953 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.954 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.954 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.954 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rgw", 2023-08-25T05:00:50.954 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.955 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.955 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.955 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.955 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.956 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.956 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.956 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.956 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.957 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.957 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.957 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.957 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.958 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.958 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.958 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.958 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.958 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.959 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.960 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.961 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.963 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.964 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.965 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.966 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.967 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.968 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.969 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rook", 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.970 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "drive_group_interval": { 2023-08-25T05:00:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "drive_group_interval", 2023-08-25T05:00:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.971 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "300.0", 2023-08-25T05:00:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.972 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-25T05:00:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.973 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.974 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.974 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.974 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.974 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.974 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.975 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.976 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.978 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.979 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.980 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.981 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.982 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.983 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.984 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.985 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.986 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "storage_class": { 2023-08-25T05:00:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "storage_class", 2023-08-25T05:00:50.987 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "local", 2023-08-25T05:00:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.988 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-25T05:00:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.989 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:50.990 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "selftest", 2023-08-25T05:00:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:50.991 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.992 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:50.993 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.994 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:50.995 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:50.996 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:50.997 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:50.998 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:50.999 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.000 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.001 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.002 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.003 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.004 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "roption1": { 2023-08-25T05:00:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption1", 2023-08-25T05:00:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.005 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.006 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.007 INFO:teuthology.orchestra.run.smithi145.stdout: "roption2": { 2023-08-25T05:00:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "roption2", 2023-08-25T05:00:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.008 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "xyz", 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.009 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption1": { 2023-08-25T05:00:51.010 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption1", 2023-08-25T05:00:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.011 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.012 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption2": { 2023-08-25T05:00:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption2", 2023-08-25T05:00:51.013 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.014 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.015 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption3": { 2023-08-25T05:00:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption3", 2023-08-25T05:00:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "float", 2023-08-25T05:00:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.016 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.017 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption4": { 2023-08-25T05:00:51.018 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption4", 2023-08-25T05:00:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.019 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.020 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption5": { 2023-08-25T05:00:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption5", 2023-08-25T05:00:51.021 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.022 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.023 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption6": { 2023-08-25T05:00:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption6", 2023-08-25T05:00:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.024 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.025 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.026 INFO:teuthology.orchestra.run.smithi145.stdout: "rwoption7": { 2023-08-25T05:00:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rwoption7", 2023-08-25T05:00:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "1", 2023-08-25T05:00:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "42", 2023-08-25T05:00:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.028 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "testkey": { 2023-08-25T05:00:51.029 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testkey", 2023-08-25T05:00:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.030 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.031 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "testlkey": { 2023-08-25T05:00:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testlkey", 2023-08-25T05:00:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.032 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.033 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "testnewline": { 2023-08-25T05:00:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "testnewline", 2023-08-25T05:00:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.035 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.036 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.037 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snap_schedule", 2023-08-25T05:00:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.038 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "allow_m_granularity": { 2023-08-25T05:00:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "allow_m_granularity", 2023-08-25T05:00:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.039 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.040 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "allow minute scheduled snapshots", 2023-08-25T05:00:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.041 INFO:teuthology.orchestra.run.smithi145.stdout: "dump_on_update": { 2023-08-25T05:00:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "dump_on_update", 2023-08-25T05:00:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.042 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "dump database to debug log on update", 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.043 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.044 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.045 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.046 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.047 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.048 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.049 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.050 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.051 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.052 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.053 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.054 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.055 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.056 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.057 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "stats", 2023-08-25T05:00:51.058 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.059 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.060 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.060 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.060 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.060 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.060 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.061 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.062 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.063 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.064 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.065 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.066 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.067 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.068 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.070 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.071 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.072 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "status", 2023-08-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.074 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.075 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.076 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.077 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.078 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.079 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.080 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.081 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.082 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.083 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.084 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.085 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.086 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.087 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.088 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telegraf", 2023-08-25T05:00:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.089 INFO:teuthology.orchestra.run.smithi145.stdout: "address": { 2023-08-25T05:00:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "address", 2023-08-25T05:00:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.090 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:51.092 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "15", 2023-08-25T05:00:51.093 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.094 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.095 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.096 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.097 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.098 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.099 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.100 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.101 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.102 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.103 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.104 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.105 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.106 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.107 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.108 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "telemetry", 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.109 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.110 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.110 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_basic": { 2023-08-25T05:00:51.110 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_basic", 2023-08-25T05:00:51.110 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.110 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.111 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.111 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:51.111 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.111 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.111 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.112 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-25T05:00:51.112 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.112 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.112 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.112 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_crash": { 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_crash", 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.113 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:51.114 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.114 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.114 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.114 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-25T05:00:51.114 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.115 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.115 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.115 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.115 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_device": { 2023-08-25T05:00:51.115 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_device", 2023-08-25T05:00:51.116 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.116 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.116 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.116 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "True", 2023-08-25T05:00:51.116 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.117 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.117 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.117 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-25T05:00:51.117 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.117 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.118 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.118 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.118 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_ident": { 2023-08-25T05:00:51.118 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_ident", 2023-08-25T05:00:51.118 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.119 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.119 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.119 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.119 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.119 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.120 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.120 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-25T05:00:51.120 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.120 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.120 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.121 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.121 INFO:teuthology.orchestra.run.smithi145.stdout: "channel_perf": { 2023-08-25T05:00:51.121 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "channel_perf", 2023-08-25T05:00:51.121 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.121 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.122 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-25T05:00:51.123 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.123 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.123 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.123 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.123 INFO:teuthology.orchestra.run.smithi145.stdout: "contact": { 2023-08-25T05:00:51.124 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "contact", 2023-08-25T05:00:51.124 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.124 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.124 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.124 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.125 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.125 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.125 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.125 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.125 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.126 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.126 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.126 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.126 INFO:teuthology.orchestra.run.smithi145.stdout: "description": { 2023-08-25T05:00:51.126 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "description", 2023-08-25T05:00:51.127 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.127 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.127 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.127 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.127 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.128 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.128 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.128 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.128 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.128 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.129 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.129 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.129 INFO:teuthology.orchestra.run.smithi145.stdout: "device_url": { 2023-08-25T05:00:51.129 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "device_url", 2023-08-25T05:00:51.129 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.130 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.131 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.131 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.131 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.131 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.131 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.132 INFO:teuthology.orchestra.run.smithi145.stdout: "enabled": { 2023-08-25T05:00:51.132 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "enabled", 2023-08-25T05:00:51.132 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.132 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.132 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.133 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.133 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.133 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.133 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.133 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.134 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.134 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.134 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.134 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.134 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:51.135 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:51.135 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.135 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.135 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.135 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "24", 2023-08-25T05:00:51.136 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "8", 2023-08-25T05:00:51.136 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.136 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.136 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.136 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.137 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.137 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.137 INFO:teuthology.orchestra.run.smithi145.stdout: "last_opt_revision": { 2023-08-25T05:00:51.137 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "last_opt_revision", 2023-08-25T05:00:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "1", 2023-08-25T05:00:51.138 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.139 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.139 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.139 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.139 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.139 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.140 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.140 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard": { 2023-08-25T05:00:51.140 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard", 2023-08-25T05:00:51.140 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.141 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.142 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.142 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.142 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.142 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.142 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.143 INFO:teuthology.orchestra.run.smithi145.stdout: "leaderboard_description": { 2023-08-25T05:00:51.143 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "leaderboard_description", 2023-08-25T05:00:51.143 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.143 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.143 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.144 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.144 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.144 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.144 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.144 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.145 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.145 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.145 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.145 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.145 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.146 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.146 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.146 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.146 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.146 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.147 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.147 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.147 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.147 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.147 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.148 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.148 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.148 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.148 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.148 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.149 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.149 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.149 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.149 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.150 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.150 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.150 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.150 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.150 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.151 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.151 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.151 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.151 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.151 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.152 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.152 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.152 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.152 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.153 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.154 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.154 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.154 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.154 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.154 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.155 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.155 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.155 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.155 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.156 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.156 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.156 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.157 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.157 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.157 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.157 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.158 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.158 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.158 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.158 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.158 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.160 INFO:teuthology.orchestra.run.smithi145.stdout: "organization": { 2023-08-25T05:00:51.160 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "organization", 2023-08-25T05:00:51.160 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.160 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.160 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.161 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.161 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.161 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.161 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.161 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.162 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.162 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.162 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.162 INFO:teuthology.orchestra.run.smithi145.stdout: "proxy": { 2023-08-25T05:00:51.163 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "proxy", 2023-08-25T05:00:51.163 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.163 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.163 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.163 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.164 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.164 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.164 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.164 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.164 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.165 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.165 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.165 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.165 INFO:teuthology.orchestra.run.smithi145.stdout: "url": { 2023-08-25T05:00:51.165 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "url", 2023-08-25T05:00:51.166 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.166 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.166 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.166 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-25T05:00:51.166 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.167 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.168 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.168 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.168 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.168 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.168 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "test_orchestrator", 2023-08-25T05:00:51.169 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.169 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.169 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.169 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.170 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.170 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.170 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.170 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.170 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.171 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.171 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.171 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.171 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.171 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.172 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.173 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.173 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.173 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.173 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.174 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.174 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.174 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.174 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.174 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.175 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.175 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.175 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.175 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.175 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.176 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.177 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.177 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.177 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.177 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.177 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.178 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.178 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.178 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.178 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.178 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.179 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.179 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.179 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.179 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.179 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.180 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.180 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.180 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.180 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.180 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.181 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.181 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.181 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.181 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.181 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.182 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.183 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.183 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.183 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.183 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.183 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.184 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "volumes", 2023-08-25T05:00:51.184 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.184 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.184 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.184 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.185 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.185 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.185 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.185 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.185 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.186 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.186 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.186 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.186 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.186 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.187 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.187 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.187 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.187 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.187 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.188 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.188 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.188 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.188 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.188 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.189 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.189 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.189 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.189 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.189 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.190 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.191 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.192 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.192 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.192 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.192 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.192 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.193 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.193 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.193 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.193 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.193 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.194 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.194 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.194 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.194 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.194 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.195 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.195 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.195 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.195 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.195 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.196 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.196 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.196 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.196 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.196 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.197 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.198 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.198 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.198 INFO:teuthology.orchestra.run.smithi145.stdout: "max_concurrent_clones": { 2023-08-25T05:00:51.198 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "max_concurrent_clones", 2023-08-25T05:00:51.198 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.199 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.199 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.199 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "4", 2023-08-25T05:00:51.199 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.199 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.200 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.200 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-25T05:00:51.200 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.200 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.200 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.201 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.201 INFO:teuthology.orchestra.run.smithi145.stdout: "snapshot_clone_delay": { 2023-08-25T05:00:51.201 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snapshot_clone_delay", 2023-08-25T05:00:51.201 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.201 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "0", 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.202 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-25T05:00:51.203 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.203 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.203 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.203 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.204 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.204 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.204 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.204 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix", 2023-08-25T05:00:51.204 INFO:teuthology.orchestra.run.smithi145.stdout: "can_run": true, 2023-08-25T05:00:51.205 INFO:teuthology.orchestra.run.smithi145.stdout: "error_string": "", 2023-08-25T05:00:51.205 INFO:teuthology.orchestra.run.smithi145.stdout: "module_options": { 2023-08-25T05:00:51.205 INFO:teuthology.orchestra.run.smithi145.stdout: "discovery_interval": { 2023-08-25T05:00:51.205 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "discovery_interval", 2023-08-25T05:00:51.205 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "uint", 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "100", 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.206 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.207 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.207 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.207 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.207 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.207 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.208 INFO:teuthology.orchestra.run.smithi145.stdout: "identifier": { 2023-08-25T05:00:51.208 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "identifier", 2023-08-25T05:00:51.208 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.208 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.208 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.209 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.209 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.209 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.209 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.209 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.210 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.210 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.210 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.210 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.210 INFO:teuthology.orchestra.run.smithi145.stdout: "interval": { 2023-08-25T05:00:51.211 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "interval", 2023-08-25T05:00:51.211 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "secs", 2023-08-25T05:00:51.211 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.211 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.211 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "60", 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.212 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.213 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.213 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.213 INFO:teuthology.orchestra.run.smithi145.stdout: "log_level": { 2023-08-25T05:00:51.213 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_level", 2023-08-25T05:00:51.213 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.214 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.214 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.214 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.214 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.214 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.215 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.215 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.215 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.215 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.215 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.216 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.216 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.216 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.216 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.216 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster": { 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster", 2023-08-25T05:00:51.217 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.218 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.218 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.218 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.218 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.218 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.219 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.219 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.219 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.219 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.219 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.220 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.220 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_cluster_level": { 2023-08-25T05:00:51.220 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_cluster_level", 2023-08-25T05:00:51.220 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.220 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.221 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.221 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "info", 2023-08-25T05:00:51.221 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.221 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.221 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [ 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "", 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "critical", 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "debug", 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "error", 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "info", 2023-08-25T05:00:51.222 INFO:teuthology.orchestra.run.smithi145.stdout: "warning" 2023-08-25T05:00:51.223 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.223 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.223 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.223 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.223 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.224 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.224 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": { 2023-08-25T05:00:51.224 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "log_to_file", 2023-08-25T05:00:51.224 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "bool", 2023-08-25T05:00:51.224 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.225 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 1, 2023-08-25T05:00:51.225 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "False", 2023-08-25T05:00:51.225 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.225 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.225 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.226 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.226 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.226 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.226 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.226 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.227 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_host": { 2023-08-25T05:00:51.227 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_host", 2023-08-25T05:00:51.227 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.227 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.227 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "", 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.228 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.229 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.229 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.229 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.229 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_port": { 2023-08-25T05:00:51.229 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_port", 2023-08-25T05:00:51.230 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "int", 2023-08-25T05:00:51.230 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.230 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.230 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "10051", 2023-08-25T05:00:51.230 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.231 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.231 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.231 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.231 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.231 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.232 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.232 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.232 INFO:teuthology.orchestra.run.smithi145.stdout: "zabbix_sender": { 2023-08-25T05:00:51.232 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "zabbix_sender", 2023-08-25T05:00:51.232 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "str", 2023-08-25T05:00:51.233 INFO:teuthology.orchestra.run.smithi145.stdout: "level": "advanced", 2023-08-25T05:00:51.233 INFO:teuthology.orchestra.run.smithi145.stdout: "flags": 0, 2023-08-25T05:00:51.233 INFO:teuthology.orchestra.run.smithi145.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-25T05:00:51.233 INFO:teuthology.orchestra.run.smithi145.stdout: "min": "", 2023-08-25T05:00:51.233 INFO:teuthology.orchestra.run.smithi145.stdout: "max": "", 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: "enum_allowed": [], 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: "desc": "", 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: "long_desc": "", 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: "tags": [], 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: "see_also": [] 2023-08-25T05:00:51.234 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.235 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.235 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.235 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.235 INFO:teuthology.orchestra.run.smithi145.stdout: "services": { 2023-08-25T05:00:51.235 INFO:teuthology.orchestra.run.smithi145.stdout: "dashboard": "https://172.21.15.145:8443/" 2023-08-25T05:00:51.236 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.236 INFO:teuthology.orchestra.run.smithi145.stdout: "always_on_modules": { 2023-08-25T05:00:51.236 INFO:teuthology.orchestra.run.smithi145.stdout: "octopus": [ 2023-08-25T05:00:51.236 INFO:teuthology.orchestra.run.smithi145.stdout: "balancer", 2023-08-25T05:00:51.236 INFO:teuthology.orchestra.run.smithi145.stdout: "crash", 2023-08-25T05:00:51.237 INFO:teuthology.orchestra.run.smithi145.stdout: "devicehealth", 2023-08-25T05:00:51.237 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator", 2023-08-25T05:00:51.237 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_autoscaler", 2023-08-25T05:00:51.237 INFO:teuthology.orchestra.run.smithi145.stdout: "progress", 2023-08-25T05:00:51.237 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_support", 2023-08-25T05:00:51.238 INFO:teuthology.orchestra.run.smithi145.stdout: "status", 2023-08-25T05:00:51.238 INFO:teuthology.orchestra.run.smithi145.stdout: "telemetry", 2023-08-25T05:00:51.238 INFO:teuthology.orchestra.run.smithi145.stdout: "volumes" 2023-08-25T05:00:51.238 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.238 INFO:teuthology.orchestra.run.smithi145.stdout: "pacific": [ 2023-08-25T05:00:51.239 INFO:teuthology.orchestra.run.smithi145.stdout: "balancer", 2023-08-25T05:00:51.239 INFO:teuthology.orchestra.run.smithi145.stdout: "crash", 2023-08-25T05:00:51.239 INFO:teuthology.orchestra.run.smithi145.stdout: "devicehealth", 2023-08-25T05:00:51.239 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator", 2023-08-25T05:00:51.239 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_autoscaler", 2023-08-25T05:00:51.240 INFO:teuthology.orchestra.run.smithi145.stdout: "progress", 2023-08-25T05:00:51.240 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_support", 2023-08-25T05:00:51.240 INFO:teuthology.orchestra.run.smithi145.stdout: "status", 2023-08-25T05:00:51.240 INFO:teuthology.orchestra.run.smithi145.stdout: "telemetry", 2023-08-25T05:00:51.240 INFO:teuthology.orchestra.run.smithi145.stdout: "volumes" 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: "quincy": [ 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: "balancer", 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: "crash", 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: "devicehealth", 2023-08-25T05:00:51.241 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator", 2023-08-25T05:00:51.242 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_autoscaler", 2023-08-25T05:00:51.242 INFO:teuthology.orchestra.run.smithi145.stdout: "progress", 2023-08-25T05:00:51.242 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_support", 2023-08-25T05:00:51.242 INFO:teuthology.orchestra.run.smithi145.stdout: "status", 2023-08-25T05:00:51.242 INFO:teuthology.orchestra.run.smithi145.stdout: "telemetry", 2023-08-25T05:00:51.243 INFO:teuthology.orchestra.run.smithi145.stdout: "volumes" 2023-08-25T05:00:51.243 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2023-08-25T05:00:51.243 INFO:teuthology.orchestra.run.smithi145.stdout: "reef": [ 2023-08-25T05:00:51.243 INFO:teuthology.orchestra.run.smithi145.stdout: "balancer", 2023-08-25T05:00:51.243 INFO:teuthology.orchestra.run.smithi145.stdout: "crash", 2023-08-25T05:00:51.244 INFO:teuthology.orchestra.run.smithi145.stdout: "devicehealth", 2023-08-25T05:00:51.244 INFO:teuthology.orchestra.run.smithi145.stdout: "orchestrator", 2023-08-25T05:00:51.244 INFO:teuthology.orchestra.run.smithi145.stdout: "pg_autoscaler", 2023-08-25T05:00:51.244 INFO:teuthology.orchestra.run.smithi145.stdout: "progress", 2023-08-25T05:00:51.244 INFO:teuthology.orchestra.run.smithi145.stdout: "rbd_support", 2023-08-25T05:00:51.245 INFO:teuthology.orchestra.run.smithi145.stdout: "status", 2023-08-25T05:00:51.245 INFO:teuthology.orchestra.run.smithi145.stdout: "telemetry", 2023-08-25T05:00:51.245 INFO:teuthology.orchestra.run.smithi145.stdout: "volumes" 2023-08-25T05:00:51.245 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.245 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: "last_failure_osd_epoch": 90, 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: "active_clients": [ 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "devicehealth", 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.246 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.247 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.247 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.247 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 2488672847 2023-08-25T05:00:51.247 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.247 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.248 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.248 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.248 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "libcephsqlite", 2023-08-25T05:00:51.248 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.248 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.249 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.249 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.249 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 1203386023 2023-08-25T05:00:51.249 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.249 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.250 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.250 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.250 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "rbd_support", 2023-08-25T05:00:51.250 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.250 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 2140522033 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.251 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.252 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.252 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snap_schedule", 2023-08-25T05:00:51.252 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.252 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.252 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.253 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.253 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 483423019 2023-08-25T05:00:51.253 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.253 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.253 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.254 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "snap_schedule", 2023-08-25T05:00:51.254 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.254 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.254 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.255 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.255 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3930926840 2023-08-25T05:00:51.255 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.255 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.255 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:51.256 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.256 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "volumes", 2023-08-25T05:00:51.256 INFO:teuthology.orchestra.run.smithi145.stdout: "addrvec": [ 2023-08-25T05:00:51.256 INFO:teuthology.orchestra.run.smithi145.stdout: { 2023-08-25T05:00:51.256 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v2", 2023-08-25T05:00:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "172.21.15.145:0", 2023-08-25T05:00:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 1446437132 2023-08-25T05:00:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.257 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:51.258 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2023-08-25T05:00:51.258 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:00:51.259 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/693177211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:00:51.260 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:46 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2132404083' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T05:00:51.260 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:47 smithi183 ceph-mon[131414]: pgmap v492: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:51.260 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:49 smithi183 ceph-mon[131414]: pgmap v493: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:51.261 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/693177211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:00:51.261 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:46 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2132404083' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T05:00:51.261 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:47 smithi162 ceph-mon[130941]: pgmap v492: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:51.261 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:49 smithi162 ceph-mon[130941]: pgmap v493: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:51.261 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 msgr2=0x7f19500a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:51.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 0x7f19500a2640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958009260 comp rx=0 tx=0).stop 2023-08-25T05:00:51.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 msgr2=0x7f196819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:00:51.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f195c005910 tx=0x7f195c004710 comp rx=0 tx=0).stop 2023-08-25T05:00:51.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 shutdown_connections 2023-08-25T05:00:51.262 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19500a0180 0x7f19500a2640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:51.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19681068a0 0x7f1968196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:51.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968100890 0x7f196819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:51.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 --2- 172.21.15.145:0/2132404083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101250 0x7f196819cc60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:00:51.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 >> 172.21.15.145:0/2132404083 conn(0x7f19680fc340 msgr2=0x7f19680fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:00:51.263 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 shutdown_connections 2023-08-25T05:00:51.264 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:00:46.639+0000 7f196cbff700 1 -- 172.21.15.145:0/2132404083 wait complete. 2023-08-25T05:00:51.270 DEBUG:tasks.check_counter:Getting stats from a 2023-08-25T05:00:51.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.a perf dump 2023-08-25T05:00:51.271 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/693177211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:00:51.272 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:46 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2132404083' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-25T05:00:51.272 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:47 smithi145 ceph-mon[97681]: pgmap v492: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:51.272 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:49 smithi145 ceph-mon[97681]: pgmap v493: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:52.074 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T05:00:52.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:51 smithi162 ceph-mon[130941]: pgmap v494: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:52.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:51 smithi183 ceph-mon[131414]: pgmap v494: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:52.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:51 smithi145 ceph-mon[97681]: pgmap v494: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:53.728 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:53.728 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:00:53.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 7345, 2023-08-25T05:00:53.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 6019, 2023-08-25T05:00:53.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3622434, 2023-08-25T05:00:53.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 5322123, 2023-08-25T05:00:53.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 9, 2023-08-25T05:00:53.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2023-08-25T05:00:53.730 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 1.054253485, 2023-08-25T05:00:53.730 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.222503986, 2023-08-25T05:00:53.730 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.312295198, 2023-08-25T05:00:53.730 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.452521199, 2023-08-25T05:00:53.730 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:53.731 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6020, 2023-08-25T05:00:53.731 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.349694407, 2023-08-25T05:00:53.731 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000058088 2023-08-25T05:00:53.731 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.731 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:53.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 10055, 2023-08-25T05:00:53.732 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.001994460, 2023-08-25T05:00:53.732 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000198 2023-08-25T05:00:53.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.732 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1412128, 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 193280 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 5510008, 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 5509280, 2023-08-25T05:00:53.733 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 871980120, 2023-08-25T05:00:53.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 936635906, 2023-08-25T05:00:53.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 20, 2023-08-25T05:00:53.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2023-08-25T05:00:53.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 255.094604790, 2023-08-25T05:00:53.734 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 114.617359970, 2023-08-25T05:00:53.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 168.926495888, 2023-08-25T05:00:53.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 1.200361997, 2023-08-25T05:00:53.735 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:53.735 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 5509281, 2023-08-25T05:00:53.735 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 81.815886891, 2023-08-25T05:00:53.736 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000014850 2023-08-25T05:00:53.736 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.736 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:53.736 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 11016065, 2023-08-25T05:00:53.736 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.194799870, 2023-08-25T05:00:53.737 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000108 2023-08-25T05:00:53.737 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.737 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 476112, 2023-08-25T05:00:53.737 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 703184 2023-08-25T05:00:53.738 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.738 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:00:53.738 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 8451, 2023-08-25T05:00:53.738 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 12340, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 2522899, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 13766412, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 86, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 8, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.826066869, 2023-08-25T05:00:53.739 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.503230044, 2023-08-25T05:00:53.740 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.681160000, 2023-08-25T05:00:53.740 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.042738663, 2023-08-25T05:00:53.740 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:53.740 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 12333, 2023-08-25T05:00:53.740 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.520931562, 2023-08-25T05:00:53.741 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000042238 2023-08-25T05:00:53.741 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.741 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:53.741 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9528, 2023-08-25T05:00:53.741 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.002863646, 2023-08-25T05:00:53.742 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000300 2023-08-25T05:00:53.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.742 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 42912, 2023-08-25T05:00:53.742 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 2128 2023-08-25T05:00:53.742 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.743 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2023-08-25T05:00:53.743 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2023-08-25T05:00:53.743 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2023-08-25T05:00:53.743 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.744 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2023-08-25T05:00:53.744 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2023-08-25T05:00:53.744 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2023-08-25T05:00:53.744 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4232, 2023-08-25T05:00:53.744 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2.835444834, 2023-08-25T05:00:53.745 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000670001 2023-08-25T05:00:53.745 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.745 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.745 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2023-08-25T05:00:53.745 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2023-08-25T05:00:53.746 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2023-08-25T05:00:53.746 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 147, 2023-08-25T05:00:53.746 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.006850599, 2023-08-25T05:00:53.746 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000046602 2023-08-25T05:00:53.746 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.747 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.747 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2023-08-25T05:00:53.747 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 2672, 2023-08-25T05:00:53.747 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 2481, 2023-08-25T05:00:53.748 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2023-08-25T05:00:53.748 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2481, 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 12.028141477, 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.004848102 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 191, 2023-08-25T05:00:53.749 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 1, 2023-08-25T05:00:53.750 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2023-08-25T05:00:53.750 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 88, 2023-08-25T05:00:53.750 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 9, 2023-08-25T05:00:53.750 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2023-08-25T05:00:53.750 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 1680, 2023-08-25T05:00:53.751 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 357, 2023-08-25T05:00:53.751 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 1137, 2023-08-25T05:00:53.751 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 186, 2023-08-25T05:00:53.751 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 1170, 2023-08-25T05:00:53.751 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 1075, 2023-08-25T05:00:53.752 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 32, 2023-08-25T05:00:53.752 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 33, 2023-08-25T05:00:53.752 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 55, 2023-08-25T05:00:53.752 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 5173, 2023-08-25T05:00:53.753 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 3748, 2023-08-25T05:00:53.753 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 171, 2023-08-25T05:00:53.753 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 53, 2023-08-25T05:00:53.753 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2023-08-25T05:00:53.753 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2023-08-25T05:00:53.754 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 467, 2023-08-25T05:00:53.754 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2023-08-25T05:00:53.754 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2023-08-25T05:00:53.754 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 36, 2023-08-25T05:00:53.754 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 923, 2023-08-25T05:00:53.755 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 27, 2023-08-25T05:00:53.755 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 590, 2023-08-25T05:00:53.755 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2023-08-25T05:00:53.755 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 1783, 2023-08-25T05:00:53.755 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2023-08-25T05:00:53.756 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 991, 2023-08-25T05:00:53.756 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 6101576512, 2023-08-25T05:00:53.756 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2023-08-25T05:00:53.756 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 1783, 2023-08-25T05:00:53.758 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2023-08-25T05:00:53.758 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:00:53.758 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:00:53.758 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 57, 2023-08-25T05:00:53.759 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 3, 2023-08-25T05:00:53.759 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 54, 2023-08-25T05:00:53.759 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 1726, 2023-08-25T05:00:53.759 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 20, 2023-08-25T05:00:53.759 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 1023, 2023-08-25T05:00:53.760 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 3587, 2023-08-25T05:00:53.760 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:00:53.760 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:00:53.760 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T05:00:53.760 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 3566, 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 1176, 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 50, 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 339 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2023-08-25T05:00:53.761 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 89, 2023-08-25T05:00:53.762 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2023-08-25T05:00:53.762 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2023-08-25T05:00:53.762 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 53, 2023-08-25T05:00:53.762 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 216, 2023-08-25T05:00:53.763 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 9, 2023-08-25T05:00:53.763 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2023-08-25T05:00:53.763 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:00:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 91, 2023-08-25T05:00:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 36, 2023-08-25T05:00:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 198, 2023-08-25T05:00:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 3, 2023-08-25T05:00:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2023-08-25T05:00:53.765 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:00:53.765 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:00:53.765 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2023-08-25T05:00:53.765 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2023-08-25T05:00:53.765 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 3, 2023-08-25T05:00:53.766 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 36, 2023-08-25T05:00:53.766 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2023-08-25T05:00:53.766 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 9, 2023-08-25T05:00:53.766 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2023-08-25T05:00:53.766 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2023-08-25T05:00:53.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.767 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2023-08-25T05:00:53.767 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2023-08-25T05:00:53.767 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 6017, 2023-08-25T05:00:53.767 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2023-08-25T05:00:53.768 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2023-08-25T05:00:53.768 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 6017, 2023-08-25T05:00:53.769 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2023-08-25T05:00:53.769 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2023-08-25T05:00:53.769 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 8, 2023-08-25T05:00:53.769 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2023-08-25T05:00:53.769 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 8, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2023-08-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 20325405, 2023-08-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2023-08-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2023-08-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2246, 2023-08-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 29.215626071, 2023-08-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.013007847 2023-08-25T05:00:53.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.772 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2023-08-25T05:00:53.772 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.772 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2023-08-25T05:00:53.772 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1543, 2023-08-25T05:00:53.773 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 2465, 2023-08-25T05:00:53.773 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 922, 2023-08-25T05:00:53.773 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 145, 2023-08-25T05:00:53.774 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 172, 2023-08-25T05:00:53.774 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 27, 2023-08-25T05:00:53.774 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 1692, 2023-08-25T05:00:53.774 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 3129, 2023-08-25T05:00:53.774 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 1437, 2023-08-25T05:00:53.775 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 33, 2023-08-25T05:00:53.775 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 1670, 2023-08-25T05:00:53.775 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 1637, 2023-08-25T05:00:53.775 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 70332, 2023-08-25T05:00:53.775 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 207152 2023-08-25T05:00:53.776 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.776 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2023-08-25T05:00:53.776 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 4148, 2023-08-25T05:00:53.776 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 335, 2023-08-25T05:00:53.776 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 2672, 2023-08-25T05:00:53.777 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 412, 2023-08-25T05:00:53.777 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 526, 2023-08-25T05:00:53.777 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2023-08-25T05:00:53.777 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1033, 2023-08-25T05:00:53.777 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 7.580672333, 2023-08-25T05:00:53.778 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.007338501 2023-08-25T05:00:53.778 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.778 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2023-08-25T05:00:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 211, 2023-08-25T05:00:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.174202033, 2023-08-25T05:00:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000825602 2023-08-25T05:00:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2023-08-25T05:00:53.780 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.780 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.780 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.780 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.781 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2023-08-25T05:00:53.781 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 57, 2023-08-25T05:00:53.781 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.221608952, 2023-08-25T05:00:53.781 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.003887876 2023-08-25T05:00:53.781 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.782 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2023-08-25T05:00:53.782 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 189, 2023-08-25T05:00:53.782 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.343704783, 2023-08-25T05:00:53.782 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001818543 2023-08-25T05:00:53.782 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.783 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2023-08-25T05:00:53.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.783 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.783 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.784 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.784 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2023-08-25T05:00:53.784 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.784 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.785 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.786 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.786 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2023-08-25T05:00:53.786 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.786 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.786 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.787 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.787 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2023-08-25T05:00:53.787 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.787 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.787 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.788 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.788 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2023-08-25T05:00:53.788 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.788 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.789 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.789 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.789 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2023-08-25T05:00:53.789 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 15, 2023-08-25T05:00:53.790 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.002615684, 2023-08-25T05:00:53.790 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000174378 2023-08-25T05:00:53.790 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.790 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2023-08-25T05:00:53.790 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 28, 2023-08-25T05:00:53.791 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.465532805, 2023-08-25T05:00:53.791 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.016626171 2023-08-25T05:00:53.791 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.791 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2023-08-25T05:00:53.792 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.792 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.792 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.792 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.792 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2023-08-25T05:00:53.793 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 15, 2023-08-25T05:00:53.793 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.077207834, 2023-08-25T05:00:53.793 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.005147188 2023-08-25T05:00:53.793 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.793 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2023-08-25T05:00:53.794 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 531, 2023-08-25T05:00:53.794 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.158327528, 2023-08-25T05:00:53.794 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000298168 2023-08-25T05:00:53.795 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.795 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2023-08-25T05:00:53.795 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 90, 2023-08-25T05:00:53.795 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.446660738, 2023-08-25T05:00:53.795 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.004962897 2023-08-25T05:00:53.796 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.796 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2023-08-25T05:00:53.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 223, 2023-08-25T05:00:53.796 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2.264343953, 2023-08-25T05:00:53.796 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.010154008 2023-08-25T05:00:53.797 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.797 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2023-08-25T05:00:53.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.797 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.797 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.798 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.798 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2023-08-25T05:00:53.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.798 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.798 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.799 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.799 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2023-08-25T05:00:53.799 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.800 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.800 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.800 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.800 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2023-08-25T05:00:53.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9, 2023-08-25T05:00:53.801 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.043230032, 2023-08-25T05:00:53.801 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.004803336 2023-08-25T05:00:53.801 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.801 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2023-08-25T05:00:53.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.802 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.802 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.802 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.802 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2023-08-25T05:00:53.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2023-08-25T05:00:53.803 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000274705, 2023-08-25T05:00:53.803 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000274705 2023-08-25T05:00:53.803 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.803 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2023-08-25T05:00:53.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.804 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.804 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.805 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.805 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2023-08-25T05:00:53.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.805 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.805 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.806 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.806 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2023-08-25T05:00:53.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.806 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.806 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.807 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.807 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2023-08-25T05:00:53.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1, 2023-08-25T05:00:53.807 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.003015321, 2023-08-25T05:00:53.807 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.003015321 2023-08-25T05:00:53.808 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.808 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2023-08-25T05:00:53.808 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.808 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.809 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.809 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.809 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2023-08-25T05:00:53.809 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 78, 2023-08-25T05:00:53.810 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.246744776, 2023-08-25T05:00:53.810 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.003163394 2023-08-25T05:00:53.810 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.810 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:00:53.811 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:00:53.811 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2023-08-25T05:00:53.811 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.811 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.811 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.812 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.812 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.812 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2023-08-25T05:00:53.812 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2023-08-25T05:00:53.813 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2023-08-25T05:00:53.813 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2023-08-25T05:00:53.813 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2023-08-25T05:00:53.813 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2023-08-25T05:00:53.813 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 28, 2023-08-25T05:00:53.814 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 14, 2023-08-25T05:00:53.814 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 916 2023-08-25T05:00:53.814 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.814 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2023-08-25T05:00:53.815 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:00:53.815 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2023-08-25T05:00:53.815 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:00:53.816 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:00:53.816 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:00:53.816 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:00:53.816 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:00:53.816 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:00:53.817 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:00:53.817 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:00:53.817 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:00:53.817 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:00:53.817 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:00:53.818 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:00:53.818 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:00:53.818 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2023-08-25T05:00:53.818 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:00:53.818 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2023-08-25T05:00:53.819 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:00:53.819 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:00:53.819 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:00:53.819 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:00:53.819 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:00:53.820 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:00:53.820 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:00:53.820 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2023-08-25T05:00:53.821 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:00:53.821 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:00:53.821 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:00:53.821 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2023-08-25T05:00:53.821 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2023-08-25T05:00:53.822 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2023-08-25T05:00:53.822 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:00:53.822 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:00:53.822 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:00:53.822 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 612779, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 1100, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 440, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 5, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2023-08-25T05:00:53.823 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2023-08-25T05:00:53.824 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:00:53.824 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2023-08-25T05:00:53.824 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:00:53.824 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2023-08-25T05:00:53.824 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9232, 2023-08-25T05:00:53.825 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 177, 2023-08-25T05:00:53.825 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:00:53.825 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:00:53.825 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2023-08-25T05:00:53.825 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2023-08-25T05:00:53.826 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 5010430, 2023-08-25T05:00:53.826 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 53347, 2023-08-25T05:00:53.826 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2023-08-25T05:00:53.826 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2023-08-25T05:00:53.826 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2023-08-25T05:00:53.827 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2023-08-25T05:00:53.827 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.827 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2023-08-25T05:00:53.827 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2023-08-25T05:00:53.827 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2023-08-25T05:00:53.828 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 4356, 2023-08-25T05:00:53.828 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 17121117, 2023-08-25T05:00:53.828 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 31, 2023-08-25T05:00:53.828 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 4356, 2023-08-25T05:00:53.828 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4356, 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 27.306433073, 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.006268694 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2023-08-25T05:00:53.829 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2023-08-25T05:00:53.830 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4356, 2023-08-25T05:00:53.830 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 4503 2023-08-25T05:00:53.830 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.830 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 4356, 2023-08-25T05:00:53.830 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 1809, 2023-08-25T05:00:53.831 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 2547, 2023-08-25T05:00:53.831 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2023-08-25T05:00:53.831 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2023-08-25T05:00:53.831 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2023-08-25T05:00:53.831 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 11, 2023-08-25T05:00:53.832 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 21, 2023-08-25T05:00:53.832 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 2264, 2023-08-25T05:00:53.832 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 53, 2023-08-25T05:00:53.832 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2023-08-25T05:00:53.832 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2023-08-25T05:00:53.833 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2023-08-25T05:00:53.833 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 4, 2023-08-25T05:00:53.833 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 92, 2023-08-25T05:00:53.833 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2023-08-25T05:00:53.833 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1786, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 22, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2023-08-25T05:00:53.834 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:00:53.835 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2023-08-25T05:00:53.835 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2023-08-25T05:00:53.835 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2023-08-25T05:00:53.835 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:00:53.835 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2023-08-25T05:00:53.836 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:00:53.837 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2023-08-25T05:00:53.837 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2023-08-25T05:00:53.837 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2023-08-25T05:00:53.837 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2023-08-25T05:00:53.837 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2023-08-25T05:00:53.838 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2023-08-25T05:00:53.838 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2023-08-25T05:00:53.838 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2023-08-25T05:00:53.838 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2023-08-25T05:00:53.838 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2023-08-25T05:00:53.839 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2023-08-25T05:00:53.840 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2023-08-25T05:00:53.840 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 108, 2023-08-25T05:00:53.840 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2023-08-25T05:00:53.840 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 26, 2023-08-25T05:00:53.841 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2023-08-25T05:00:53.841 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 12, 2023-08-25T05:00:53.842 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2023-08-25T05:00:53.842 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2023-08-25T05:00:53.842 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 222, 2023-08-25T05:00:53.842 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 3, 2023-08-25T05:00:53.842 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 25 2023-08-25T05:00:53.843 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.843 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2023-08-25T05:00:53.844 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2023-08-25T05:00:53.844 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:00:53.844 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2023-08-25T05:00:53.844 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2023-08-25T05:00:53.844 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.845 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2023-08-25T05:00:53.845 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2023-08-25T05:00:53.845 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.845 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2023-08-25T05:00:53.845 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 16, 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 8, 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 36, 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2023-08-25T05:00:53.846 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.847 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:00:53.847 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:53.847 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2023-08-25T05:00:53.848 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:53.848 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 5525801, 2023-08-25T05:00:53.848 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 380828743, 2023-08-25T05:00:53.848 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:53.848 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 5525801, 2023-08-25T05:00:53.849 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:00:53.849 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:00:53.849 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 5525801, 2023-08-25T05:00:53.849 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 380828743, 2023-08-25T05:00:53.849 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:53.850 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.850 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.850 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.850 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.850 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.851 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2023-08-25T05:00:53.851 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:53.851 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2023-08-25T05:00:53.851 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:53.851 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2023-08-25T05:00:53.852 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2023-08-25T05:00:53.852 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:53.852 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2023-08-25T05:00:53.852 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 4356, 2023-08-25T05:00:53.852 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 17125063, 2023-08-25T05:00:53.853 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 4356, 2023-08-25T05:00:53.853 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 17125063, 2023-08-25T05:00:53.853 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:53.853 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.853 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.854 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.854 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.854 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.854 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2023-08-25T05:00:53.854 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:53.855 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2023-08-25T05:00:53.856 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 4356, 2023-08-25T05:00:53.856 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 4356, 2023-08-25T05:00:53.856 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 4356, 2023-08-25T05:00:53.856 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4356, 2023-08-25T05:00:53.856 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:53.857 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.857 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.857 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.857 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.857 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.858 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:00:53.858 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:53.858 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2023-08-25T05:00:53.858 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:53.858 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 36, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 3988, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 36, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:00:53.859 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 18, 2023-08-25T05:00:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 3988, 2023-08-25T05:00:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.860 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.861 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.861 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:53.861 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55bce63fc6e0": { 2023-08-25T05:00:53.861 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:53.861 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2023-08-25T05:00:53.862 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:53.862 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 6017, 2023-08-25T05:00:53.862 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 16131101, 2023-08-25T05:00:53.862 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:53.863 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 6017, 2023-08-25T05:00:53.863 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:00:53.863 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:00:53.863 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2246, 2023-08-25T05:00:53.863 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 16131101, 2023-08-25T05:00:53.864 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:53.864 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:53.864 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:53.864 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:53.864 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.865 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:53.865 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:00:54.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:53 smithi145 ceph-mon[97681]: pgmap v495: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:54.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:53 smithi183 ceph-mon[131414]: pgmap v495: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:54.213 INFO:tasks.check_counter:Daemon mds.a mds.exported=36 2023-08-25T05:00:54.214 INFO:tasks.check_counter:Daemon mds.a mds.imported=27 2023-08-25T05:00:54.214 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=89 2023-08-25T05:00:54.214 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-08-25T05:00:54.215 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2023-08-25T05:00:54.215 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=15 2023-08-25T05:00:54.215 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=9 2023-08-25T05:00:54.215 DEBUG:tasks.check_counter:Getting stats from d 2023-08-25T05:00:54.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.d perf dump 2023-08-25T05:00:54.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:53 smithi162 ceph-mon[130941]: pgmap v495: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:54.991 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T05:00:56.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:55 smithi183 ceph-mon[131414]: pgmap v496: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:56.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:55 smithi145 ceph-mon[97681]: pgmap v496: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:56.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:55 smithi162 ceph-mon[130941]: pgmap v496: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:00:56.656 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:56.656 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:00:56.657 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 311, 2023-08-25T05:00:56.657 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 300, 2023-08-25T05:00:56.657 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1304416, 2023-08-25T05:00:56.657 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 179568, 2023-08-25T05:00:56.657 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2023-08-25T05:00:56.658 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2023-08-25T05:00:56.658 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.056556378, 2023-08-25T05:00:56.658 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.018848648, 2023-08-25T05:00:56.658 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.037892350, 2023-08-25T05:00:56.658 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.005956928, 2023-08-25T05:00:56.659 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:56.659 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 300, 2023-08-25T05:00:56.659 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.016367130, 2023-08-25T05:00:56.659 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000054557 2023-08-25T05:00:56.659 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.660 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:56.660 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:56.660 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:56.660 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:56.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.661 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1304416, 2023-08-25T05:00:56.661 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 179568 2023-08-25T05:00:56.661 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.662 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:00:56.662 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3, 2023-08-25T05:00:56.662 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 228, 2023-08-25T05:00:56.662 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 480, 2023-08-25T05:00:56.662 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 269040, 2023-08-25T05:00:56.663 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2023-08-25T05:00:56.663 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2023-08-25T05:00:56.663 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.021099770, 2023-08-25T05:00:56.663 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.013403979, 2023-08-25T05:00:56.664 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.003870100, 2023-08-25T05:00:56.664 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:00:56.664 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:56.665 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 225, 2023-08-25T05:00:56.665 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.017201777, 2023-08-25T05:00:56.665 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000076452 2023-08-25T05:00:56.666 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.666 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:56.666 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:56.667 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:56.667 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:56.667 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.667 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T05:00:56.668 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 269040 2023-08-25T05:00:56.668 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.668 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:00:56.669 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 0, 2023-08-25T05:00:56.669 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2, 2023-08-25T05:00:56.669 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:00:56.670 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 0, 2023-08-25T05:00:56.670 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2023-08-25T05:00:56.670 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2023-08-25T05:00:56.670 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.001106903, 2023-08-25T05:00:56.670 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:00:56.671 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.000961728, 2023-08-25T05:00:56.671 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:00:56.671 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:56.671 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:56.671 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:56.672 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:56.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.672 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:56.672 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:56.673 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:56.673 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:56.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.673 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:00:56.673 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:00:56.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.674 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2023-08-25T05:00:56.674 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2023-08-25T05:00:56.674 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2023-08-25T05:00:56.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.675 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2023-08-25T05:00:56.675 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:00:56.675 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2023-08-25T05:00:56.675 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:00:56.675 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:00:56.676 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:00:56.676 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:00:56.676 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:00:56.676 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:00:56.677 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:00:56.677 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:00:56.677 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:00:56.677 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:00:56.677 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:00:56.678 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:00:56.678 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:00:56.678 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2023-08-25T05:00:56.678 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:00:56.678 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2023-08-25T05:00:56.679 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:00:56.679 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:00:56.679 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:00:56.679 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:00:56.679 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:00:56.680 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:00:56.680 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:00:56.680 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2023-08-25T05:00:56.680 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:00:56.680 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:00:56.681 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:00:56.681 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2023-08-25T05:00:56.681 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2023-08-25T05:00:56.681 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2023-08-25T05:00:56.681 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:00:56.682 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:00:56.682 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:00:56.682 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:00:56.682 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148731, 2023-08-25T05:00:56.682 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 31, 2023-08-25T05:00:56.683 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:00:56.683 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2023-08-25T05:00:56.683 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2023-08-25T05:00:56.683 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2023-08-25T05:00:56.683 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:00:56.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2023-08-25T05:00:56.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:00:56.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2023-08-25T05:00:56.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2023-08-25T05:00:56.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2023-08-25T05:00:56.685 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:00:56.685 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:00:56.685 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2023-08-25T05:00:56.685 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2023-08-25T05:00:56.685 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2023-08-25T05:00:56.686 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2023-08-25T05:00:56.686 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2023-08-25T05:00:56.686 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2023-08-25T05:00:56.686 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2023-08-25T05:00:56.686 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2023-08-25T05:00:56.687 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:56.687 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:00:56.687 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:56.687 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2023-08-25T05:00:56.688 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:56.688 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 310, 2023-08-25T05:00:56.688 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1258084, 2023-08-25T05:00:56.688 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:56.688 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 310, 2023-08-25T05:00:56.689 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:00:56.689 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:00:56.689 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 310, 2023-08-25T05:00:56.689 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1258084, 2023-08-25T05:00:56.689 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:56.690 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:56.690 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:56.690 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:56.690 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:56.690 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:56.691 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:00:57.220 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-08-25T05:00:57.221 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-08-25T05:00:57.221 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-08-25T05:00:57.221 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-08-25T05:00:57.222 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-08-25T05:00:57.222 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-08-25T05:00:57.222 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-08-25T05:00:57.222 DEBUG:tasks.check_counter:Getting stats from g 2023-08-25T05:00:57.223 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.g perf dump 2023-08-25T05:00:57.998 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T05:00:58.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:57 smithi183 ceph-mon[131414]: pgmap v497: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:58.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:57 smithi145 ceph-mon[97681]: pgmap v497: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:58.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:57 smithi162 ceph-mon[130941]: pgmap v497: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-08-25T05:00:59.751 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:00:59.752 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:00:59.752 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 0, 2023-08-25T05:00:59.752 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2, 2023-08-25T05:00:59.753 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:00:59.753 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 0, 2023-08-25T05:00:59.753 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2023-08-25T05:00:59.753 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2023-08-25T05:00:59.753 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.003459831, 2023-08-25T05:00:59.754 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:00:59.754 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.000705796, 2023-08-25T05:00:59.754 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:00:59.754 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:59.754 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:59.755 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:59.755 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:59.755 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.755 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:59.755 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:59.756 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:59.756 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:59.756 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.756 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:00:59.756 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:00:59.757 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.757 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:00:59.757 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 3, 2023-08-25T05:00:59.757 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 225, 2023-08-25T05:00:59.758 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 480, 2023-08-25T05:00:59.758 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 268512, 2023-08-25T05:00:59.758 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2023-08-25T05:00:59.758 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2023-08-25T05:00:59.759 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.019886901, 2023-08-25T05:00:59.759 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.012838649, 2023-08-25T05:00:59.759 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.003582926, 2023-08-25T05:00:59.759 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:00:59.759 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:59.760 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 222, 2023-08-25T05:00:59.760 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.020421687, 2023-08-25T05:00:59.760 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000091989 2023-08-25T05:00:59.760 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.760 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:59.761 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:59.761 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:59.761 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:59.761 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.761 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 268512 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 308, 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 297, 2023-08-25T05:00:59.762 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1303600, 2023-08-25T05:00:59.763 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 177696, 2023-08-25T05:00:59.763 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2023-08-25T05:00:59.763 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2023-08-25T05:00:59.763 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.052577688, 2023-08-25T05:00:59.763 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.018444714, 2023-08-25T05:00:59.764 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.036943982, 2023-08-25T05:00:59.764 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.005809555, 2023-08-25T05:00:59.764 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:00:59.764 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 297, 2023-08-25T05:00:59.764 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.014111151, 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000047512 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:59.765 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:59.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.766 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1303600, 2023-08-25T05:00:59.766 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 177696 2023-08-25T05:00:59.766 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.766 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2023-08-25T05:00:59.767 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2023-08-25T05:00:59.767 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2023-08-25T05:00:59.767 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.767 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2023-08-25T05:00:59.767 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:00:59.768 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2023-08-25T05:00:59.768 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:00:59.769 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:00:59.769 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:00:59.769 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:00:59.769 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:00:59.769 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:00:59.770 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:00:59.770 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:00:59.770 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:00:59.770 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:00:59.770 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2023-08-25T05:00:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:00:59.772 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:00:59.772 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:00:59.772 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:00:59.772 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:00:59.772 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:00:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:00:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2023-08-25T05:00:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:00:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:00:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:00:59.774 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2023-08-25T05:00:59.774 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2023-08-25T05:00:59.774 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2023-08-25T05:00:59.774 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:00:59.774 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:00:59.775 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:00:59.775 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:00:59.775 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148714, 2023-08-25T05:00:59.775 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 30, 2023-08-25T05:00:59.775 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:00:59.776 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2023-08-25T05:00:59.776 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2023-08-25T05:00:59.776 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2023-08-25T05:00:59.776 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:00:59.776 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2023-08-25T05:00:59.777 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:00:59.777 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2023-08-25T05:00:59.777 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2023-08-25T05:00:59.777 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2023-08-25T05:00:59.777 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2023-08-25T05:00:59.778 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2023-08-25T05:00:59.779 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2023-08-25T05:00:59.779 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2023-08-25T05:00:59.779 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2023-08-25T05:00:59.779 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:00:59.779 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:00:59.780 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:00:59.780 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2023-08-25T05:00:59.780 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:00:59.780 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 307, 2023-08-25T05:00:59.780 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1257694, 2023-08-25T05:00:59.781 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:00:59.781 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 307, 2023-08-25T05:00:59.781 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:00:59.781 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:00:59.781 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 307, 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1257694, 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:00:59.782 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:59.783 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:00:59.783 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:01:00.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:00:59 smithi145 ceph-mon[97681]: pgmap v498: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:01:00.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:00:59 smithi183 ceph-mon[131414]: pgmap v498: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:01:00.225 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-08-25T05:01:00.226 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-08-25T05:01:00.226 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-08-25T05:01:00.226 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-08-25T05:01:00.227 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-08-25T05:01:00.227 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-08-25T05:01:00.227 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-08-25T05:01:00.227 DEBUG:tasks.check_counter:Getting stats from j 2023-08-25T05:01:00.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.j perf dump 2023-08-25T05:01:00.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:00:59 smithi162 ceph-mon[130941]: pgmap v498: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-08-25T05:01:01.034 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.a/config 2023-08-25T05:01:02.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:01 smithi145 ceph-mon[97681]: pgmap v499: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:01:02.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:01 smithi162 ceph-mon[130941]: pgmap v499: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:01:02.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:01 smithi183 ceph-mon[131414]: pgmap v499: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-08-25T05:01:02.689 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-08-25T05:01:02.690 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:02.690 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 309, 2023-08-25T05:01:02.690 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 300, 2023-08-25T05:01:02.690 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1301968, 2023-08-25T05:01:02.691 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 180368, 2023-08-25T05:01:02.691 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2023-08-25T05:01:02.691 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2023-08-25T05:01:02.691 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.056628038, 2023-08-25T05:01:02.691 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.019206349, 2023-08-25T05:01:02.692 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.038014053, 2023-08-25T05:01:02.692 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.006139591, 2023-08-25T05:01:02.692 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:02.692 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 299, 2023-08-25T05:01:02.692 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.016631600, 2023-08-25T05:01:02.693 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000055624 2023-08-25T05:01:02.693 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.693 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:02.693 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:01:02.693 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1301968, 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 180368 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.694 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:02.695 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 7, 2023-08-25T05:01:02.695 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 231, 2023-08-25T05:01:02.695 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3472, 2023-08-25T05:01:02.695 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 269664, 2023-08-25T05:01:02.695 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2023-08-25T05:01:02.696 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2023-08-25T05:01:02.696 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.021706189, 2023-08-25T05:01:02.696 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.014359046, 2023-08-25T05:01:02.696 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.004076002, 2023-08-25T05:01:02.696 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:02.697 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:02.697 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 229, 2023-08-25T05:01:02.697 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.018525647, 2023-08-25T05:01:02.697 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000080898 2023-08-25T05:01:02.697 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.698 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3472, 2023-08-25T05:01:02.699 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 269664 2023-08-25T05:01:02.699 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.699 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:02.699 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 0, 2023-08-25T05:01:02.699 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2, 2023-08-25T05:01:02.700 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:01:02.700 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 0, 2023-08-25T05:01:02.700 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2023-08-25T05:01:02.700 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2023-08-25T05:01:02.700 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.000648375, 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.000485822, 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:01:02.701 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:01:02.702 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:01:02.702 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.702 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:02.702 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:01:02.702 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:01:02.703 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:01:02.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.703 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:02.703 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:02.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.704 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2023-08-25T05:01:02.704 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2023-08-25T05:01:02.704 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2023-08-25T05:01:02.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.704 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:02.705 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:02.706 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:02.706 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:02.706 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:02.706 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:02.706 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:02.707 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:02.707 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:02.707 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:02.707 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:02.707 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:02.708 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:02.709 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:02.709 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:02.709 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:02.709 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:02.709 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:02.710 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:02.710 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:02.710 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:02.710 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:02.710 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148714, 2023-08-25T05:01:02.711 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 30, 2023-08-25T05:01:02.712 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:02.712 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:02.712 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2023-08-25T05:01:02.712 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2023-08-25T05:01:02.712 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:02.713 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:02.713 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:02.713 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:02.713 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2023-08-25T05:01:02.713 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2023-08-25T05:01:02.714 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:02.714 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:02.714 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:02.714 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2023-08-25T05:01:02.714 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2023-08-25T05:01:02.715 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2023-08-25T05:01:02.716 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:02.716 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2023-08-25T05:01:02.716 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2023-08-25T05:01:02.716 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2023-08-25T05:01:02.716 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 312, 2023-08-25T05:01:02.717 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1258344, 2023-08-25T05:01:02.717 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:02.717 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 312, 2023-08-25T05:01:02.717 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2023-08-25T05:01:02.717 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2023-08-25T05:01:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 312, 2023-08-25T05:01:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1258344, 2023-08-25T05:01:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2023-08-25T05:01:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2023-08-25T05:01:02.718 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2023-08-25T05:01:02.719 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2023-08-25T05:01:02.719 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:01:02.719 INFO:teuthology.orchestra.run.smithi145.stdout: } 2023-08-25T05:01:02.719 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-08-25T05:01:03.165 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-08-25T05:01:03.166 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-08-25T05:01:03.166 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-08-25T05:01:03.166 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-08-25T05:01:03.167 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-08-25T05:01:03.167 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-08-25T05:01:03.167 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-08-25T05:01:03.167 DEBUG:tasks.check_counter:Getting stats from b 2023-08-25T05:01:03.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.b perf dump 2023-08-25T05:01:03.748 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T05:01:04.193 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:03 smithi162 ceph-mon[130941]: pgmap v500: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:04.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:03 smithi145 ceph-mon[97681]: pgmap v500: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:04.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:03 smithi183 ceph-mon[131414]: pgmap v500: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:05.084 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-08-25T05:01:05.085 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:05.085 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 5623, 2023-08-25T05:01:05.085 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 7174, 2023-08-25T05:01:05.086 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1854436, 2023-08-25T05:01:05.086 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 6287335, 2023-08-25T05:01:05.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 10, 2023-08-25T05:01:05.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2023-08-25T05:01:05.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.343853834, 2023-08-25T05:01:05.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.139956291, 2023-08-25T05:01:05.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.180734467, 2023-08-25T05:01:05.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.032266100, 2023-08-25T05:01:05.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:05.090 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 7173, 2023-08-25T05:01:05.090 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.181674401, 2023-08-25T05:01:05.091 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000025327 2023-08-25T05:01:05.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:05.091 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 10075, 2023-08-25T05:01:05.092 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.001753761, 2023-08-25T05:01:05.092 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000174 2023-08-25T05:01:05.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.093 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 6912, 2023-08-25T05:01:05.093 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 447712 2023-08-25T05:01:05.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.093 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:05.093 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 3703, 2023-08-25T05:01:05.094 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 3822, 2023-08-25T05:01:05.094 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1153179, 2023-08-25T05:01:05.094 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 2043196, 2023-08-25T05:01:05.094 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 19, 2023-08-25T05:01:05.095 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 9, 2023-08-25T05:01:05.095 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.266420795, 2023-08-25T05:01:05.095 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.080099209, 2023-08-25T05:01:05.095 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.135159574, 2023-08-25T05:01:05.095 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.068291973, 2023-08-25T05:01:05.096 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:05.096 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3817, 2023-08-25T05:01:05.096 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.086503194, 2023-08-25T05:01:05.096 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000022662 2023-08-25T05:01:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2931, 2023-08-25T05:01:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000976167, 2023-08-25T05:01:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000333 2023-08-25T05:01:05.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.098 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1776, 2023-08-25T05:01:05.098 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 123792 2023-08-25T05:01:05.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.099 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:05.099 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2396, 2023-08-25T05:01:05.099 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1772, 2023-08-25T05:01:05.099 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 2201172, 2023-08-25T05:01:05.100 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 680424, 2023-08-25T05:01:05.100 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 9, 2023-08-25T05:01:05.100 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 6, 2023-08-25T05:01:05.101 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.138724432, 2023-08-25T05:01:05.101 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.050784072, 2023-08-25T05:01:05.101 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.098879667, 2023-08-25T05:01:05.102 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.010284179, 2023-08-25T05:01:05.102 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:05.102 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1773, 2023-08-25T05:01:05.103 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.059240801, 2023-08-25T05:01:05.103 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000033412 2023-08-25T05:01:05.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.104 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:05.104 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3224, 2023-08-25T05:01:05.104 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000414701, 2023-08-25T05:01:05.104 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000128 2023-08-25T05:01:05.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.105 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1399808, 2023-08-25T05:01:05.105 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 186800 2023-08-25T05:01:05.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.106 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2023-08-25T05:01:05.106 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2023-08-25T05:01:05.107 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2023-08-25T05:01:05.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2023-08-25T05:01:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2023-08-25T05:01:05.108 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2023-08-25T05:01:05.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2117, 2023-08-25T05:01:05.109 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1.006695182, 2023-08-25T05:01:05.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000475529 2023-08-25T05:01:05.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.111 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2023-08-25T05:01:05.111 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2023-08-25T05:01:05.111 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2023-08-25T05:01:05.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 112, 2023-08-25T05:01:05.112 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.003105672, 2023-08-25T05:01:05.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000027729 2023-08-25T05:01:05.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.114 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2023-08-25T05:01:05.114 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 2017, 2023-08-25T05:01:05.114 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 1950, 2023-08-25T05:01:05.114 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2023-08-25T05:01:05.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1950, 2023-08-25T05:01:05.115 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 10.171474288, 2023-08-25T05:01:05.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.005216140 2023-08-25T05:01:05.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.116 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2023-08-25T05:01:05.116 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 67, 2023-08-25T05:01:05.117 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 3, 2023-08-25T05:01:05.117 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2023-08-25T05:01:05.117 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 20, 2023-08-25T05:01:05.117 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 1, 2023-08-25T05:01:05.117 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2023-08-25T05:01:05.118 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 677, 2023-08-25T05:01:05.118 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 31, 2023-08-25T05:01:05.118 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 507, 2023-08-25T05:01:05.118 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 139, 2023-08-25T05:01:05.118 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 633, 2023-08-25T05:01:05.119 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 334, 2023-08-25T05:01:05.119 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 9, 2023-08-25T05:01:05.119 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 10, 2023-08-25T05:01:05.120 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 23, 2023-08-25T05:01:05.120 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 12247, 2023-08-25T05:01:05.121 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 8868, 2023-08-25T05:01:05.121 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 55, 2023-08-25T05:01:05.121 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 62, 2023-08-25T05:01:05.122 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 1, 2023-08-25T05:01:05.122 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2023-08-25T05:01:05.123 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 2599, 2023-08-25T05:01:05.123 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2023-08-25T05:01:05.123 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2023-08-25T05:01:05.123 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 11, 2023-08-25T05:01:05.123 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 235, 2023-08-25T05:01:05.124 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 23, 2023-08-25T05:01:05.124 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 544, 2023-08-25T05:01:05.124 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2023-08-25T05:01:05.124 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 65, 2023-08-25T05:01:05.125 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2023-08-25T05:01:05.125 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 991, 2023-08-25T05:01:05.125 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 6101576512, 2023-08-25T05:01:05.125 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 3, 2023-08-25T05:01:05.126 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 65, 2023-08-25T05:01:05.126 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2023-08-25T05:01:05.126 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:01:05.126 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:01:05.126 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 1, 2023-08-25T05:01:05.127 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T05:01:05.127 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 1, 2023-08-25T05:01:05.127 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 64, 2023-08-25T05:01:05.127 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 35, 2023-08-25T05:01:05.127 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 157, 2023-08-25T05:01:05.128 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 455, 2023-08-25T05:01:05.128 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:01:05.128 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:01:05.129 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 1, 2023-08-25T05:01:05.129 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 631, 2023-08-25T05:01:05.129 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 447, 2023-08-25T05:01:05.130 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 51, 2023-08-25T05:01:05.130 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 854 2023-08-25T05:01:05.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.131 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2023-08-25T05:01:05.131 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 34, 2023-08-25T05:01:05.131 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 172, 2023-08-25T05:01:05.132 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 100, 2023-08-25T05:01:05.132 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 153, 2023-08-25T05:01:05.133 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 84, 2023-08-25T05:01:05.133 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 15, 2023-08-25T05:01:05.133 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2023-08-25T05:01:05.134 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:01:05.134 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 209, 2023-08-25T05:01:05.135 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 11, 2023-08-25T05:01:05.135 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 826, 2023-08-25T05:01:05.135 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 2, 2023-08-25T05:01:05.136 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2023-08-25T05:01:05.136 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:01:05.136 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:01:05.136 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2023-08-25T05:01:05.136 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2023-08-25T05:01:05.137 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T05:01:05.137 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 11, 2023-08-25T05:01:05.137 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2023-08-25T05:01:05.137 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 1, 2023-08-25T05:01:05.137 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 1558, 2023-08-25T05:01:05.138 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2023-08-25T05:01:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2023-08-25T05:01:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 1558, 2023-08-25T05:01:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2023-08-25T05:01:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2023-08-25T05:01:05.139 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 3, 2023-08-25T05:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2023-08-25T05:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2023-08-25T05:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 3, 2023-08-25T05:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 2, 2023-08-25T05:01:05.140 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2023-08-25T05:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2023-08-25T05:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 4194304, 2023-08-25T05:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 7643253, 2023-08-25T05:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 0, 2023-08-25T05:01:05.141 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1045, 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1.972214711, 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001887286 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 0 2023-08-25T05:01:05.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2023-08-25T05:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 415, 2023-08-25T05:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 522, 2023-08-25T05:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 107, 2023-08-25T05:01:05.143 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 92, 2023-08-25T05:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 161, 2023-08-25T05:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 69, 2023-08-25T05:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 693, 2023-08-25T05:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 1093, 2023-08-25T05:01:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 400, 2023-08-25T05:01:05.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 10, 2023-08-25T05:01:05.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 346, 2023-08-25T05:01:05.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 336, 2023-08-25T05:01:05.145 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 58012, 2023-08-25T05:01:05.145 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 198960 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 7753, 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 248, 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 2017, 2023-08-25T05:01:05.146 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 405, 2023-08-25T05:01:05.147 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 484, 2023-08-25T05:01:05.147 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2023-08-25T05:01:05.147 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 38, 2023-08-25T05:01:05.147 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.053165864, 2023-08-25T05:01:05.147 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001399101 2023-08-25T05:01:05.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.148 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2023-08-25T05:01:05.148 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 43, 2023-08-25T05:01:05.148 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.101548947, 2023-08-25T05:01:05.148 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.002361603 2023-08-25T05:01:05.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.149 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2023-08-25T05:01:05.149 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.149 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.149 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 204, 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.496364991, 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.002433161 2023-08-25T05:01:05.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.151 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2023-08-25T05:01:05.151 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 512, 2023-08-25T05:01:05.151 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.613660578, 2023-08-25T05:01:05.151 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001198555 2023-08-25T05:01:05.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2023-08-25T05:01:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.153 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2023-08-25T05:01:05.153 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.153 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.153 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.154 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2023-08-25T05:01:05.155 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.155 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.155 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.155 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2023-08-25T05:01:05.156 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.156 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.156 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.156 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2023-08-25T05:01:05.157 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.157 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.157 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.157 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2023-08-25T05:01:05.158 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.158 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.158 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.158 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2023-08-25T05:01:05.159 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.159 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.159 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.159 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2023-08-25T05:01:05.160 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.160 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.160 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.160 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 84, 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.111558080, 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001328072 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2023-08-25T05:01:05.161 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 33, 2023-08-25T05:01:05.162 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.028142031, 2023-08-25T05:01:05.162 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000852788 2023-08-25T05:01:05.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.162 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2023-08-25T05:01:05.162 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 829, 2023-08-25T05:01:05.163 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 8.320482964, 2023-08-25T05:01:05.163 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.010036770 2023-08-25T05:01:05.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.163 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2023-08-25T05:01:05.163 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.164 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.164 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.164 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2023-08-25T05:01:05.164 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.165 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.165 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.165 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2023-08-25T05:01:05.165 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.166 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.166 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.166 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2023-08-25T05:01:05.167 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.167 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.167 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.167 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2023-08-25T05:01:05.168 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.168 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.168 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.169 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2023-08-25T05:01:05.169 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.169 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.169 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.170 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2023-08-25T05:01:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.171 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2023-08-25T05:01:05.172 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.172 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.172 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.172 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2023-08-25T05:01:05.173 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.173 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.173 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.173 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2023-08-25T05:01:05.174 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.174 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.174 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.174 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 207, 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.446550833, 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.002157250 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:01:05.175 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:01:05.176 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2023-08-25T05:01:05.176 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.176 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.176 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.177 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2023-08-25T05:01:05.177 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 2, 2023-08-25T05:01:05.177 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 3, 2023-08-25T05:01:05.177 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 1, 2023-08-25T05:01:05.178 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 2, 2023-08-25T05:01:05.178 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2023-08-25T05:01:05.178 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 522, 2023-08-25T05:01:05.178 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 261, 2023-08-25T05:01:05.178 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 940 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:05.179 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:05.180 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:05.181 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:05.181 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:05.181 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:05.181 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:05.181 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:05.182 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:05.182 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:05.182 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:05.182 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:05.182 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:05.183 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:05.183 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:05.183 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:05.183 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:05.183 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:05.184 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:05.184 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:05.184 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:05.184 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:05.184 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:05.185 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 826612, 2023-08-25T05:01:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 696, 2023-08-25T05:01:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 88, 2023-08-25T05:01:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 1, 2023-08-25T05:01:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2023-08-25T05:01:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2023-08-25T05:01:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 9232, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 177, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2023-08-25T05:01:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 1743981, 2023-08-25T05:01:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 32409, 2023-08-25T05:01:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2023-08-25T05:01:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2023-08-25T05:01:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2023-08-25T05:01:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2023-08-25T05:01:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2023-08-25T05:01:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 1287, 2023-08-25T05:01:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 3670081, 2023-08-25T05:01:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 27, 2023-08-25T05:01:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 1287, 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1287, 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 2.681943083, 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.002083871 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2023-08-25T05:01:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2023-08-25T05:01:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1287, 2023-08-25T05:01:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1368 2023-08-25T05:01:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 1287, 2023-08-25T05:01:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 82, 2023-08-25T05:01:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 1205, 2023-08-25T05:01:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2023-08-25T05:01:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2023-08-25T05:01:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 0, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 10, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 11, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 1056, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 30, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2023-08-25T05:01:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2023-08-25T05:01:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 6, 2023-08-25T05:01:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 1, 2023-08-25T05:01:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 49, 2023-08-25T05:01:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2023-08-25T05:01:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2023-08-25T05:01:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2023-08-25T05:01:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 65, 2023-08-25T05:01:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 20, 2023-08-25T05:01:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:01:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2023-08-25T05:01:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:01:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2023-08-25T05:01:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2023-08-25T05:01:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2023-08-25T05:01:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2023-08-25T05:01:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2023-08-25T05:01:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2023-08-25T05:01:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2023-08-25T05:01:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2023-08-25T05:01:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2023-08-25T05:01:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2023-08-25T05:01:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2023-08-25T05:01:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2023-08-25T05:01:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2023-08-25T05:01:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2023-08-25T05:01:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 108, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2023-08-25T05:01:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 21, 2023-08-25T05:01:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 12, 2023-08-25T05:01:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 12, 2023-08-25T05:01:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2023-08-25T05:01:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2023-08-25T05:01:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 90, 2023-08-25T05:01:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 9, 2023-08-25T05:01:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 21 2023-08-25T05:01:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2023-08-25T05:01:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 1, 2023-08-25T05:01:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:01:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2023-08-25T05:01:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2023-08-25T05:01:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 10, 2023-08-25T05:01:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2023-08-25T05:01:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 2, 2023-08-25T05:01:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 11, 2023-08-25T05:01:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2023-08-25T05:01:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 11718, 2023-08-25T05:01:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 4189062, 2023-08-25T05:01:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 11718, 2023-08-25T05:01:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 11718, 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 4189062, 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2023-08-25T05:01:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 1287, 2023-08-25T05:01:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 3671121, 2023-08-25T05:01:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 1287, 2023-08-25T05:01:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 3671121, 2023-08-25T05:01:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2023-08-25T05:01:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2023-08-25T05:01:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 1287, 2023-08-25T05:01:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 1287, 2023-08-25T05:01:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 1287, 2023-08-25T05:01:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1287, 2023-08-25T05:01:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 11, 2023-08-25T05:01:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1223, 2023-08-25T05:01:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 11, 2023-08-25T05:01:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 11, 2023-08-25T05:01:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1223, 2023-08-25T05:01:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x5587e1ba66e0": { 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2023-08-25T05:01:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 1558, 2023-08-25T05:01:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 3448949, 2023-08-25T05:01:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 1558, 2023-08-25T05:01:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 1045, 2023-08-25T05:01:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 3448949, 2023-08-25T05:01:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:05.224 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-08-25T05:01:05.500 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2023-08-25T05:01:05.501 INFO:tasks.check_counter:Daemon mds.b mds.imported=23 2023-08-25T05:01:05.501 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=34 2023-08-25T05:01:05.501 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=172 2023-08-25T05:01:05.502 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=3 2023-08-25T05:01:05.502 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-08-25T05:01:05.502 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T05:01:05.503 DEBUG:tasks.check_counter:Getting stats from e 2023-08-25T05:01:05.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.e perf dump 2023-08-25T05:01:06.060 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T05:01:06.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:05 smithi162 ceph-mon[130941]: pgmap v501: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:06.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:05 smithi145 ceph-mon[97681]: pgmap v501: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:06.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:05 smithi183 ceph-mon[131414]: pgmap v501: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-08-25T05:01:07.289 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-08-25T05:01:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 309, 2023-08-25T05:01:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 300, 2023-08-25T05:01:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1301968, 2023-08-25T05:01:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 180352, 2023-08-25T05:01:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2023-08-25T05:01:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2023-08-25T05:01:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.056749929, 2023-08-25T05:01:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.019533896, 2023-08-25T05:01:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.032460351, 2023-08-25T05:01:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.008494184, 2023-08-25T05:01:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 299, 2023-08-25T05:01:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.015764757, 2023-08-25T05:01:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000052724 2023-08-25T05:01:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1301968, 2023-08-25T05:01:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 180352 2023-08-25T05:01:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 7, 2023-08-25T05:01:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 231, 2023-08-25T05:01:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 3472, 2023-08-25T05:01:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 269664, 2023-08-25T05:01:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2023-08-25T05:01:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.016306664, 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.010815932, 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.002271958, 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 229, 2023-08-25T05:01:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.024330566, 2023-08-25T05:01:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000106247 2023-08-25T05:01:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 3472, 2023-08-25T05:01:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 269664 2023-08-25T05:01:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2023-08-25T05:01:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2023-08-25T05:01:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:01:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2023-08-25T05:01:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2023-08-25T05:01:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2023-08-25T05:01:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.000435637, 2023-08-25T05:01:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:01:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000358091, 2023-08-25T05:01:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2023-08-25T05:01:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2023-08-25T05:01:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2023-08-25T05:01:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2023-08-25T05:01:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2023-08-25T05:01:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148731, 2023-08-25T05:01:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 31, 2023-08-25T05:01:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2023-08-25T05:01:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2023-08-25T05:01:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2023-08-25T05:01:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2023-08-25T05:01:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2023-08-25T05:01:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2023-08-25T05:01:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 312, 2023-08-25T05:01:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1258344, 2023-08-25T05:01:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 312, 2023-08-25T05:01:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 312, 2023-08-25T05:01:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1258344, 2023-08-25T05:01:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:07.322 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-08-25T05:01:07.322 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:06 smithi162 ceph-mon[130941]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T05:01:07.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:06 smithi145 ceph-mon[97681]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T05:01:07.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:06 smithi183 ceph-mon[131414]: osdmap e109: 12 total, 12 up, 12 in 2023-08-25T05:01:07.620 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2023-08-25T05:01:07.620 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2023-08-25T05:01:07.621 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2023-08-25T05:01:07.622 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2023-08-25T05:01:07.622 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-08-25T05:01:07.622 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-08-25T05:01:07.622 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-08-25T05:01:07.623 DEBUG:tasks.check_counter:Getting stats from h 2023-08-25T05:01:07.623 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.h perf dump 2023-08-25T05:01:08.174 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T05:01:08.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:07 smithi145 ceph-mon[97681]: pgmap v503: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:08.404 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:07 smithi162 ceph-mon[130941]: pgmap v503: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:08.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:07 smithi183 ceph-mon[131414]: pgmap v503: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:09.535 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-08-25T05:01:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2378, 2023-08-25T05:01:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2355, 2023-08-25T05:01:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 6184892, 2023-08-25T05:01:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 815130, 2023-08-25T05:01:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2023-08-25T05:01:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2023-08-25T05:01:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.261168276, 2023-08-25T05:01:09.824 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.094105157, 2023-08-25T05:01:09.824 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.075132717, 2023-08-25T05:01:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.076884574, 2023-08-25T05:01:09.825 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2355, 2023-08-25T05:01:09.826 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.097609864, 2023-08-25T05:01:09.827 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000041447 2023-08-25T05:01:09.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.827 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:09.828 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.828 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.828 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.829 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1392992, 2023-08-25T05:01:09.830 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 185040 2023-08-25T05:01:09.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.831 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:09.831 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1031, 2023-08-25T05:01:09.831 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1256, 2023-08-25T05:01:09.832 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 8013141, 2023-08-25T05:01:09.832 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 864245, 2023-08-25T05:01:09.832 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 7, 2023-08-25T05:01:09.832 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2023-08-25T05:01:09.832 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.098099967, 2023-08-25T05:01:09.833 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.031882883, 2023-08-25T05:01:09.833 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.026920365, 2023-08-25T05:01:09.833 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.028705631, 2023-08-25T05:01:09.833 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:09.833 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1253, 2023-08-25T05:01:09.834 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.052317718, 2023-08-25T05:01:09.834 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000041753 2023-08-25T05:01:09.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.834 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:09.835 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.835 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.835 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.835 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-25T05:01:09.836 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 544160 2023-08-25T05:01:09.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.836 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:09.836 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 326, 2023-08-25T05:01:09.836 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 328, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 4276622, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 101393, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.028087128, 2023-08-25T05:01:09.837 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.006353559, 2023-08-25T05:01:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.009837983, 2023-08-25T05:01:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.009500485, 2023-08-25T05:01:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 326, 2023-08-25T05:01:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.009267121, 2023-08-25T05:01:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000028426 2023-08-25T05:01:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2023-08-25T05:01:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2023-08-25T05:01:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2023-08-25T05:01:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 5645, 2023-08-25T05:01:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.158124845, 2023-08-25T05:01:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000028011 2023-08-25T05:01:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2023-08-25T05:01:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2023-08-25T05:01:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2023-08-25T05:01:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2023-08-25T05:01:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 0, 2023-08-25T05:01:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 0, 2023-08-25T05:01:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2023-08-25T05:01:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2023-08-25T05:01:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 0, 2023-08-25T05:01:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2023-08-25T05:01:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2023-08-25T05:01:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 0, 2023-08-25T05:01:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 0, 2023-08-25T05:01:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2023-08-25T05:01:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 1703, 2023-08-25T05:01:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 80, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 491, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 1132, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 1588, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 30, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2023-08-25T05:01:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2023-08-25T05:01:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 53, 2023-08-25T05:01:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 0, 2023-08-25T05:01:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 0, 2023-08-25T05:01:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 0, 2023-08-25T05:01:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 0, 2023-08-25T05:01:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2023-08-25T05:01:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2023-08-25T05:01:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2023-08-25T05:01:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2023-08-25T05:01:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2023-08-25T05:01:09.851 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 0, 2023-08-25T05:01:09.851 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 0, 2023-08-25T05:01:09.851 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 0, 2023-08-25T05:01:09.851 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 0, 2023-08-25T05:01:09.851 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2023-08-25T05:01:09.852 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 0, 2023-08-25T05:01:09.852 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2023-08-25T05:01:09.852 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 991, 2023-08-25T05:01:09.852 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 6101576512, 2023-08-25T05:01:09.852 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 6, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 0, 2023-08-25T05:01:09.853 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T05:01:09.854 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T05:01:09.854 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 0, 2023-08-25T05:01:09.854 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2023-08-25T05:01:09.854 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T05:01:09.854 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 0, 2023-08-25T05:01:09.855 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:01:09.855 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:01:09.855 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T05:01:09.855 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 0, 2023-08-25T05:01:09.855 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 0, 2023-08-25T05:01:09.856 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 0, 2023-08-25T05:01:09.856 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 0 2023-08-25T05:01:09.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.856 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2023-08-25T05:01:09.856 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2023-08-25T05:01:09.857 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2023-08-25T05:01:09.857 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2023-08-25T05:01:09.857 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 0, 2023-08-25T05:01:09.857 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 0, 2023-08-25T05:01:09.857 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2023-08-25T05:01:09.858 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2023-08-25T05:01:09.859 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2023-08-25T05:01:09.859 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:01:09.859 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:01:09.859 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2023-08-25T05:01:09.859 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2023-08-25T05:01:09.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T05:01:09.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 0, 2023-08-25T05:01:09.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2023-08-25T05:01:09.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 0, 2023-08-25T05:01:09.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2023-08-25T05:01:09.861 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2023-08-25T05:01:09.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.861 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2023-08-25T05:01:09.861 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2023-08-25T05:01:09.861 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 0, 2023-08-25T05:01:09.862 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2023-08-25T05:01:09.862 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2023-08-25T05:01:09.862 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 6026, 2023-08-25T05:01:09.862 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2023-08-25T05:01:09.862 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2023-08-25T05:01:09.863 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 0, 2023-08-25T05:01:09.863 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2023-08-25T05:01:09.863 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2023-08-25T05:01:09.863 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 9, 2023-08-25T05:01:09.863 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 2, 2023-08-25T05:01:09.864 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2023-08-25T05:01:09.864 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2023-08-25T05:01:09.864 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 4194304, 2023-08-25T05:01:09.864 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 20328711, 2023-08-25T05:01:09.864 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 20328651, 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.865 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 6026 2023-08-25T05:01:09.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.866 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2023-08-25T05:01:09.866 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 1530, 2023-08-25T05:01:09.866 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 2440, 2023-08-25T05:01:09.866 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 910, 2023-08-25T05:01:09.867 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 137, 2023-08-25T05:01:09.867 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 161, 2023-08-25T05:01:09.867 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 24, 2023-08-25T05:01:09.867 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 1703, 2023-08-25T05:01:09.867 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 2690, 2023-08-25T05:01:09.868 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 987, 2023-08-25T05:01:09.868 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2023-08-25T05:01:09.868 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 0, 2023-08-25T05:01:09.868 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 0, 2023-08-25T05:01:09.868 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 55112, 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 207152 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 0, 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 0, 2023-08-25T05:01:09.869 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 0, 2023-08-25T05:01:09.870 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 0, 2023-08-25T05:01:09.870 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 0, 2023-08-25T05:01:09.870 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2023-08-25T05:01:09.870 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.870 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.871 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.871 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2023-08-25T05:01:09.871 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.871 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.872 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.872 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2023-08-25T05:01:09.872 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.872 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.873 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.874 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2023-08-25T05:01:09.874 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.874 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.874 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2023-08-25T05:01:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.875 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.877 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2023-08-25T05:01:09.877 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.877 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.877 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.878 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2023-08-25T05:01:09.878 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.878 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.878 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.879 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2023-08-25T05:01:09.879 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.879 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.879 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.880 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2023-08-25T05:01:09.881 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.881 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.881 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.881 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2023-08-25T05:01:09.882 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.882 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.882 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.882 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2023-08-25T05:01:09.883 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.883 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.883 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.883 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2023-08-25T05:01:09.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.885 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.885 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.885 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2023-08-25T05:01:09.885 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.886 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.886 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.886 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2023-08-25T05:01:09.886 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.887 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.887 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2023-08-25T05:01:09.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.888 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.888 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.888 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2023-08-25T05:01:09.888 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.889 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.890 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.890 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2023-08-25T05:01:09.890 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.890 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.891 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.891 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2023-08-25T05:01:09.891 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.891 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.892 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.892 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2023-08-25T05:01:09.892 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.892 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.893 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.893 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2023-08-25T05:01:09.893 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.893 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.894 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.894 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2023-08-25T05:01:09.894 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.894 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.895 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.895 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2023-08-25T05:01:09.895 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.895 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.896 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.896 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2023-08-25T05:01:09.896 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.896 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.897 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.897 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2023-08-25T05:01:09.897 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.897 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.898 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.898 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:01:09.898 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:01:09.898 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.899 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2023-08-25T05:01:09.900 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 2, 2023-08-25T05:01:09.900 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 4, 2023-08-25T05:01:09.900 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 2, 2023-08-25T05:01:09.900 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 3, 2023-08-25T05:01:09.900 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2023-08-25T05:01:09.901 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 0, 2023-08-25T05:01:09.901 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 0, 2023-08-25T05:01:09.901 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 3862 2023-08-25T05:01:09.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.901 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2023-08-25T05:01:09.902 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:09.902 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:09.902 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:09.902 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:09.902 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:09.903 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:09.903 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:09.903 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:09.903 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:09.903 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:09.904 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:09.904 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:09.904 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:09.904 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:09.904 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:09.905 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:09.905 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:09.905 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:09.905 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:09.905 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:09.906 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:09.906 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:09.906 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:09.906 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:09.906 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2023-08-25T05:01:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:09.908 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:09.909 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 249913, 2023-08-25T05:01:09.909 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 1668, 2023-08-25T05:01:09.909 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:09.909 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:09.909 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2023-08-25T05:01:09.910 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2023-08-25T05:01:09.910 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:09.910 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:09.910 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:09.911 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:09.911 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 9472, 2023-08-25T05:01:09.911 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 183, 2023-08-25T05:01:09.911 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:09.912 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:09.912 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:09.912 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2023-08-25T05:01:09.912 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 4969714, 2023-08-25T05:01:09.912 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 50949, 2023-08-25T05:01:09.913 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:09.913 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2023-08-25T05:01:09.913 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:09.913 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2023-08-25T05:01:09.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.914 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2023-08-25T05:01:09.914 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2023-08-25T05:01:09.914 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2023-08-25T05:01:09.914 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 3381, 2023-08-25T05:01:09.914 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 26, 2023-08-25T05:01:09.915 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 4, 2023-08-25T05:01:09.915 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 3381, 2023-08-25T05:01:09.915 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2023-08-25T05:01:09.915 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3381, 2023-08-25T05:01:09.916 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1.484968327, 2023-08-25T05:01:09.916 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000439209 2023-08-25T05:01:09.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.916 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2023-08-25T05:01:09.916 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2023-08-25T05:01:09.917 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3381, 2023-08-25T05:01:09.917 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 3382 2023-08-25T05:01:09.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.917 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 3381, 2023-08-25T05:01:09.917 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 3381, 2023-08-25T05:01:09.918 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 0, 2023-08-25T05:01:09.918 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2023-08-25T05:01:09.918 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2023-08-25T05:01:09.918 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 2142, 2023-08-25T05:01:09.918 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 0, 2023-08-25T05:01:09.919 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 1236, 2023-08-25T05:01:09.919 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 0, 2023-08-25T05:01:09.919 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 0, 2023-08-25T05:01:09.919 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2023-08-25T05:01:09.919 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2023-08-25T05:01:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2023-08-25T05:01:09.921 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 2, 2023-08-25T05:01:09.921 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 0, 2023-08-25T05:01:09.921 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:01:09.921 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2023-08-25T05:01:09.921 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:01:09.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2023-08-25T05:01:09.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2023-08-25T05:01:09.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2023-08-25T05:01:09.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:01:09.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2023-08-25T05:01:09.923 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2023-08-25T05:01:09.924 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2023-08-25T05:01:09.924 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2023-08-25T05:01:09.924 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2023-08-25T05:01:09.924 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2023-08-25T05:01:09.924 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2023-08-25T05:01:09.925 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2023-08-25T05:01:09.925 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2023-08-25T05:01:09.925 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2023-08-25T05:01:09.925 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2023-08-25T05:01:09.925 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 109, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 22, 2023-08-25T05:01:09.926 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 5, 2023-08-25T05:01:09.927 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 6, 2023-08-25T05:01:09.927 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 1, 2023-08-25T05:01:09.927 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2023-08-25T05:01:09.927 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 0, 2023-08-25T05:01:09.927 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 2, 2023-08-25T05:01:09.928 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 0 2023-08-25T05:01:09.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.928 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2023-08-25T05:01:09.928 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 0, 2023-08-25T05:01:09.928 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2023-08-25T05:01:09.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.930 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2023-08-25T05:01:09.930 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2023-08-25T05:01:09.930 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T05:01:09.930 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2023-08-25T05:01:09.930 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 0, 2023-08-25T05:01:09.931 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 0, 2023-08-25T05:01:09.931 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2023-08-25T05:01:09.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.931 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:09.931 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 3731, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 18118017, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:09.932 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 3731, 2023-08-25T05:01:09.933 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:09.933 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:09.933 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3731, 2023-08-25T05:01:09.933 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 18118017, 2023-08-25T05:01:09.933 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:09.934 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.934 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.934 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:09.935 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:09.936 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:09.936 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:09.936 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 3381, 2023-08-25T05:01:09.936 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 16134417, 2023-08-25T05:01:09.936 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3381, 2023-08-25T05:01:09.937 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 16134417, 2023-08-25T05:01:09.937 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:09.937 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.937 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.937 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.938 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2023-08-25T05:01:09.938 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:09.938 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:09.939 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 3381, 2023-08-25T05:01:09.940 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 3381, 2023-08-25T05:01:09.940 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3381, 2023-08-25T05:01:09.940 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 3381, 2023-08-25T05:01:09.940 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:09.940 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.941 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.941 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.941 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:09.942 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:09.943 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:09.943 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:09.943 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:09.943 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2023-08-25T05:01:09.943 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2023-08-25T05:01:09.944 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:09.944 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.944 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.944 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:09.945 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x559f131ba960": { 2023-08-25T05:01:09.945 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:09.945 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2023-08-25T05:01:09.945 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:09.946 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:09.947 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2023-08-25T05:01:09.947 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2023-08-25T05:01:09.947 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:09.947 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:09.947 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:09.948 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:09.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:09.948 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-08-25T05:01:09.950 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-08-25T05:01:09.950 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-08-25T05:01:09.950 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-08-25T05:01:09.950 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-08-25T05:01:09.951 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-08-25T05:01:09.951 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-08-25T05:01:09.951 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T05:01:09.951 DEBUG:tasks.check_counter:Getting stats from k 2023-08-25T05:01:09.952 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.k perf dump 2023-08-25T05:01:10.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:09 smithi145 ceph-mon[97681]: pgmap v504: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:10.411 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:09 smithi162 ceph-mon[130941]: pgmap v504: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:10.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:09 smithi183 ceph-mon[131414]: pgmap v504: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-08-25T05:01:10.508 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.b/config 2023-08-25T05:01:11.749 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-08-25T05:01:11.750 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:11.750 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2023-08-25T05:01:11.750 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2023-08-25T05:01:11.750 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:01:11.751 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2023-08-25T05:01:11.751 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2023-08-25T05:01:11.751 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2023-08-25T05:01:11.751 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001140925, 2023-08-25T05:01:11.751 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:01:11.752 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000268260, 2023-08-25T05:01:11.752 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:11.752 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:11.752 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:11.752 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:11.753 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:11.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.754 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:11.754 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:11.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.754 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:11.755 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 3, 2023-08-25T05:01:11.755 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 232, 2023-08-25T05:01:11.755 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 480, 2023-08-25T05:01:11.755 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 269744, 2023-08-25T05:01:11.755 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2023-08-25T05:01:11.756 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2023-08-25T05:01:11.756 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.015435920, 2023-08-25T05:01:11.756 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.010350992, 2023-08-25T05:01:11.756 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001732567, 2023-08-25T05:01:11.756 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:11.757 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:11.757 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 229, 2023-08-25T05:01:11.757 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.023891935, 2023-08-25T05:01:11.757 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000104331 2023-08-25T05:01:11.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.758 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:11.758 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:11.758 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:11.758 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:11.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.759 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T05:01:11.759 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 269744 2023-08-25T05:01:11.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.759 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:11.760 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 316, 2023-08-25T05:01:11.760 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 305, 2023-08-25T05:01:11.760 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1305776, 2023-08-25T05:01:11.760 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 182672, 2023-08-25T05:01:11.760 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2023-08-25T05:01:11.761 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2023-08-25T05:01:11.761 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.057765893, 2023-08-25T05:01:11.761 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.020308599, 2023-08-25T05:01:11.761 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.032618627, 2023-08-25T05:01:11.761 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.008910455, 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 305, 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.017202968, 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000056403 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.762 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:11.763 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:11.763 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:11.763 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:11.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.763 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1305776, 2023-08-25T05:01:11.764 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 182672 2023-08-25T05:01:11.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.764 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2023-08-25T05:01:11.764 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2023-08-25T05:01:11.764 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2023-08-25T05:01:11.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.765 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2023-08-25T05:01:11.765 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:11.765 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:11.765 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:11.766 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:11.767 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:11.767 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:11.767 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:11.767 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:11.767 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:11.768 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:11.768 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:11.768 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:11.768 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:11.768 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:11.769 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:11.770 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:11.770 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:11.770 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:11.771 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:11.771 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:11.771 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:11.771 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2023-08-25T05:01:11.771 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:11.772 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:11.772 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:11.772 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:11.772 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148771, 2023-08-25T05:01:11.772 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 32, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:11.773 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:11.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:11.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:11.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2023-08-25T05:01:11.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2023-08-25T05:01:11.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:11.775 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:11.775 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:11.775 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2023-08-25T05:01:11.775 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2023-08-25T05:01:11.775 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2023-08-25T05:01:11.776 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:11.776 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2023-08-25T05:01:11.776 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:11.776 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2023-08-25T05:01:11.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-08-25T05:01:11.777 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:11.777 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2023-08-25T05:01:11.777 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2023-08-25T05:01:11.777 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2023-08-25T05:01:11.777 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 315, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1258734, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 315, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2023-08-25T05:01:11.778 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 315, 2023-08-25T05:01:11.779 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1258734, 2023-08-25T05:01:11.779 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2023-08-25T05:01:11.779 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2023-08-25T05:01:11.779 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2023-08-25T05:01:11.779 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2023-08-25T05:01:11.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:11.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-08-25T05:01:11.780 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-08-25T05:01:12.097 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:11 smithi162 ceph-mon[130941]: pgmap v505: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:12.148 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-08-25T05:01:12.149 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-08-25T05:01:12.149 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-08-25T05:01:12.150 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-08-25T05:01:12.150 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-08-25T05:01:12.151 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-08-25T05:01:12.151 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-08-25T05:01:12.151 DEBUG:tasks.check_counter:Getting stats from c 2023-08-25T05:01:12.152 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.c perf dump 2023-08-25T05:01:12.327 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:11 smithi145 ceph-mon[97681]: pgmap v505: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:12.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:11 smithi183 ceph-mon[131414]: pgmap v505: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:12.737 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T05:01:13.995 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-08-25T05:01:13.995 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:13.996 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3229, 2023-08-25T05:01:13.996 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3229, 2023-08-25T05:01:13.996 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4359970, 2023-08-25T05:01:13.996 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1003430, 2023-08-25T05:01:13.997 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2023-08-25T05:01:13.997 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2023-08-25T05:01:13.997 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.196533707, 2023-08-25T05:01:13.999 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.055662292, 2023-08-25T05:01:13.999 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.048545358, 2023-08-25T05:01:14.000 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.077891248, 2023-08-25T05:01:14.000 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:14.001 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3228, 2023-08-25T05:01:14.002 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.066682141, 2023-08-25T05:01:14.002 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000020657 2023-08-25T05:01:14.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.002 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:14.002 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.003 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.003 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-25T05:01:14.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T05:01:14.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.006 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:14.006 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 350, 2023-08-25T05:01:14.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 329, 2023-08-25T05:01:14.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1390668, 2023-08-25T05:01:14.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 185740, 2023-08-25T05:01:14.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2023-08-25T05:01:14.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2023-08-25T05:01:14.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.043817468, 2023-08-25T05:01:14.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.013706967, 2023-08-25T05:01:14.009 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.030200594, 2023-08-25T05:01:14.009 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006542958, 2023-08-25T05:01:14.009 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:14.009 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 328, 2023-08-25T05:01:14.009 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.011562856, 2023-08-25T05:01:14.010 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000035252 2023-08-25T05:01:14.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.010 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:14.010 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.010 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.011 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1390112, 2023-08-25T05:01:14.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 185056 2023-08-25T05:01:14.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.012 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:14.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3, 2023-08-25T05:01:14.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 230, 2023-08-25T05:01:14.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 588, 2023-08-25T05:01:14.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 546391, 2023-08-25T05:01:14.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2023-08-25T05:01:14.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2023-08-25T05:01:14.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.010363341, 2023-08-25T05:01:14.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.006210643, 2023-08-25T05:01:14.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002139882, 2023-08-25T05:01:14.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000031200, 2023-08-25T05:01:14.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:14.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 227, 2023-08-25T05:01:14.017 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.014640824, 2023-08-25T05:01:14.017 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000064497 2023-08-25T05:01:14.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:14.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.018 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-25T05:01:14.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 546080 2023-08-25T05:01:14.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.021 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2023-08-25T05:01:14.021 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2023-08-25T05:01:14.022 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2023-08-25T05:01:14.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.023 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2023-08-25T05:01:14.023 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:14.023 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:14.024 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5458, 2023-08-25T05:01:14.024 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.115932048, 2023-08-25T05:01:14.025 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000021240 2023-08-25T05:01:14.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.026 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2023-08-25T05:01:14.026 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:14.027 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:14.027 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.027 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.028 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.029 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2023-08-25T05:01:14.029 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2023-08-25T05:01:14.030 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2023-08-25T05:01:14.030 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2023-08-25T05:01:14.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.031 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.032 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2023-08-25T05:01:14.033 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2023-08-25T05:01:14.033 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2023-08-25T05:01:14.033 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2023-08-25T05:01:14.034 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2023-08-25T05:01:14.034 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2023-08-25T05:01:14.035 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2023-08-25T05:01:14.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 687, 2023-08-25T05:01:14.037 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 6, 2023-08-25T05:01:14.037 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 342, 2023-08-25T05:01:14.038 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 339, 2023-08-25T05:01:14.038 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 678, 2023-08-25T05:01:14.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 14, 2023-08-25T05:01:14.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2023-08-25T05:01:14.041 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2023-08-25T05:01:14.041 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 21, 2023-08-25T05:01:14.042 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2023-08-25T05:01:14.043 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2023-08-25T05:01:14.043 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2023-08-25T05:01:14.044 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2023-08-25T05:01:14.044 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2023-08-25T05:01:14.044 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2023-08-25T05:01:14.044 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2023-08-25T05:01:14.045 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2023-08-25T05:01:14.047 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2023-08-25T05:01:14.049 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2023-08-25T05:01:14.050 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2023-08-25T05:01:14.051 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2023-08-25T05:01:14.051 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2023-08-25T05:01:14.052 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2023-08-25T05:01:14.052 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2023-08-25T05:01:14.053 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2023-08-25T05:01:14.053 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2023-08-25T05:01:14.054 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2023-08-25T05:01:14.054 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2023-08-25T05:01:14.055 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T05:01:14.055 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2023-08-25T05:01:14.056 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:01:14.056 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:01:14.057 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2023-08-25T05:01:14.057 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T05:01:14.058 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T05:01:14.058 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2023-08-25T05:01:14.058 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2023-08-25T05:01:14.059 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T05:01:14.059 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2023-08-25T05:01:14.060 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:01:14.060 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:01:14.061 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T05:01:14.061 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2023-08-25T05:01:14.062 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2023-08-25T05:01:14.062 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2023-08-25T05:01:14.062 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2023-08-25T05:01:14.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.063 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2023-08-25T05:01:14.064 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2023-08-25T05:01:14.064 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2023-08-25T05:01:14.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2023-08-25T05:01:14.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2023-08-25T05:01:14.066 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2023-08-25T05:01:14.066 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2023-08-25T05:01:14.066 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2023-08-25T05:01:14.067 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:01:14.067 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2023-08-25T05:01:14.068 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2023-08-25T05:01:14.068 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2023-08-25T05:01:14.069 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2023-08-25T05:01:14.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2023-08-25T05:01:14.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:01:14.070 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:01:14.070 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2023-08-25T05:01:14.070 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2023-08-25T05:01:14.071 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T05:01:14.071 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2023-08-25T05:01:14.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2023-08-25T05:01:14.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2023-08-25T05:01:14.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2023-08-25T05:01:14.073 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2023-08-25T05:01:14.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.074 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2023-08-25T05:01:14.074 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2023-08-25T05:01:14.074 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2023-08-25T05:01:14.075 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2023-08-25T05:01:14.076 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2023-08-25T05:01:14.076 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1558, 2023-08-25T05:01:14.076 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2023-08-25T05:01:14.077 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2023-08-25T05:01:14.078 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2023-08-25T05:01:14.078 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2023-08-25T05:01:14.078 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2023-08-25T05:01:14.079 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 3, 2023-08-25T05:01:14.079 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2023-08-25T05:01:14.080 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2023-08-25T05:01:14.080 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2023-08-25T05:01:14.081 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2023-08-25T05:01:14.081 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 7643253, 2023-08-25T05:01:14.081 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 7642679, 2023-08-25T05:01:14.082 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2023-08-25T05:01:14.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.082 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.084 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 1558 2023-08-25T05:01:14.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.084 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2023-08-25T05:01:14.085 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 369, 2023-08-25T05:01:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 499, 2023-08-25T05:01:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 130, 2023-08-25T05:01:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 30, 2023-08-25T05:01:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 47, 2023-08-25T05:01:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 17, 2023-08-25T05:01:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 687, 2023-08-25T05:01:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 974, 2023-08-25T05:01:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 287, 2023-08-25T05:01:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2023-08-25T05:01:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2023-08-25T05:01:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2023-08-25T05:01:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 50056, 2023-08-25T05:01:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 207152 2023-08-25T05:01:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2023-08-25T05:01:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2023-08-25T05:01:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2023-08-25T05:01:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2023-08-25T05:01:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2023-08-25T05:01:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2023-08-25T05:01:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2023-08-25T05:01:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2023-08-25T05:01:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2023-08-25T05:01:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2023-08-25T05:01:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2023-08-25T05:01:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2023-08-25T05:01:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2023-08-25T05:01:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2023-08-25T05:01:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2023-08-25T05:01:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2023-08-25T05:01:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2023-08-25T05:01:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2023-08-25T05:01:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.120 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2023-08-25T05:01:14.120 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.120 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2023-08-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2023-08-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2023-08-25T05:01:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2023-08-25T05:01:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2023-08-25T05:01:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2023-08-25T05:01:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2023-08-25T05:01:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2023-08-25T05:01:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2023-08-25T05:01:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2023-08-25T05:01:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2023-08-25T05:01:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2023-08-25T05:01:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2023-08-25T05:01:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2023-08-25T05:01:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2023-08-25T05:01:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2023-08-25T05:01:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:01:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:01:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2023-08-25T05:01:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2023-08-25T05:01:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2023-08-25T05:01:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2023-08-25T05:01:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2023-08-25T05:01:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2023-08-25T05:01:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2023-08-25T05:01:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2023-08-25T05:01:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2023-08-25T05:01:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3920 2023-08-25T05:01:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2023-08-25T05:01:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2023-08-25T05:01:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 198858, 2023-08-25T05:01:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 502, 2023-08-25T05:01:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2023-08-25T05:01:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2023-08-25T05:01:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9472, 2023-08-25T05:01:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 183, 2023-08-25T05:01:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2023-08-25T05:01:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 1462498, 2023-08-25T05:01:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 24547, 2023-08-25T05:01:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2023-08-25T05:01:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2023-08-25T05:01:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2023-08-25T05:01:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2023-08-25T05:01:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2023-08-25T05:01:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3228, 2023-08-25T05:01:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2023-08-25T05:01:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2023-08-25T05:01:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3228, 2023-08-25T05:01:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2023-08-25T05:01:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3228, 2023-08-25T05:01:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.730954404, 2023-08-25T05:01:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000536231 2023-08-25T05:01:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2023-08-25T05:01:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2023-08-25T05:01:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3228, 2023-08-25T05:01:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3229 2023-08-25T05:01:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3228, 2023-08-25T05:01:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3228, 2023-08-25T05:01:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2023-08-25T05:01:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2023-08-25T05:01:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2023-08-25T05:01:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2063, 2023-08-25T05:01:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2023-08-25T05:01:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1163, 2023-08-25T05:01:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2023-08-25T05:01:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2023-08-25T05:01:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2023-08-25T05:01:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2023-08-25T05:01:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2023-08-25T05:01:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2023-08-25T05:01:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2023-08-25T05:01:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2023-08-25T05:01:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2023-08-25T05:01:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2023-08-25T05:01:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2023-08-25T05:01:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2023-08-25T05:01:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:01:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2023-08-25T05:01:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:01:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2023-08-25T05:01:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2023-08-25T05:01:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2023-08-25T05:01:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:01:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2023-08-25T05:01:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:01:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2023-08-25T05:01:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2023-08-25T05:01:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2023-08-25T05:01:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2023-08-25T05:01:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2023-08-25T05:01:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2023-08-25T05:01:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2023-08-25T05:01:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2023-08-25T05:01:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2023-08-25T05:01:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2023-08-25T05:01:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2023-08-25T05:01:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2023-08-25T05:01:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2023-08-25T05:01:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2023-08-25T05:01:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2023-08-25T05:01:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2023-08-25T05:01:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2023-08-25T05:01:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 109, 2023-08-25T05:01:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2023-08-25T05:01:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 22, 2023-08-25T05:01:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 4, 2023-08-25T05:01:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2023-08-25T05:01:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2023-08-25T05:01:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2023-08-25T05:01:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2023-08-25T05:01:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2023-08-25T05:01:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2023-08-25T05:01:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2023-08-25T05:01:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2023-08-25T05:01:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:01:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2023-08-25T05:01:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2023-08-25T05:01:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2023-08-25T05:01:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2023-08-25T05:01:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2023-08-25T05:01:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2023-08-25T05:01:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T05:01:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2023-08-25T05:01:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2023-08-25T05:01:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2023-08-25T05:01:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2023-08-25T05:01:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3578, 2023-08-25T05:01:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5408352, 2023-08-25T05:01:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3578, 2023-08-25T05:01:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3578, 2023-08-25T05:01:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5408352, 2023-08-25T05:01:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2023-08-25T05:01:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3228, 2023-08-25T05:01:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3448954, 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3228, 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3448954, 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2023-08-25T05:01:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2023-08-25T05:01:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3228, 2023-08-25T05:01:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3228, 2023-08-25T05:01:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3228, 2023-08-25T05:01:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3228, 2023-08-25T05:01:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:01:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2023-08-25T05:01:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2023-08-25T05:01:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55b589012960": { 2023-08-25T05:01:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:14.204 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:14.204 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2023-08-25T05:01:14.204 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2023-08-25T05:01:14.204 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:14.206 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-08-25T05:01:14.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:13 smithi183 ceph-mon[131414]: pgmap v506: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:14.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:13 smithi145 ceph-mon[97681]: pgmap v506: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:14.400 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-08-25T05:01:14.400 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-08-25T05:01:14.400 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-08-25T05:01:14.401 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-08-25T05:01:14.401 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-08-25T05:01:14.401 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-08-25T05:01:14.402 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T05:01:14.402 DEBUG:tasks.check_counter:Getting stats from f 2023-08-25T05:01:14.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.f perf dump 2023-08-25T05:01:14.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:13 smithi162 ceph-mon[130941]: pgmap v506: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:14.967 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T05:01:15.922 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:15 smithi183 ceph-mon[131414]: pgmap v507: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:15.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:15 smithi183 ceph-mon[131414]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T05:01:16.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:15 smithi162 ceph-mon[130941]: pgmap v507: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:16.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:15 smithi162 ceph-mon[130941]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T05:01:16.262 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-08-25T05:01:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1157, 2023-08-25T05:01:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1159, 2023-08-25T05:01:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4129249, 2023-08-25T05:01:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 359891, 2023-08-25T05:01:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2023-08-25T05:01:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2023-08-25T05:01:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.076662926, 2023-08-25T05:01:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020635130, 2023-08-25T05:01:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.021925667, 2023-08-25T05:01:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.026103420, 2023-08-25T05:01:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1157, 2023-08-25T05:01:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027455691, 2023-08-25T05:01:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000023730 2023-08-25T05:01:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 974, 2023-08-25T05:01:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1202, 2023-08-25T05:01:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 239454, 2023-08-25T05:01:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 850121, 2023-08-25T05:01:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2023-08-25T05:01:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2023-08-25T05:01:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.071308554, 2023-08-25T05:01:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.021376525, 2023-08-25T05:01:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.015278841, 2023-08-25T05:01:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.027398521, 2023-08-25T05:01:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1199, 2023-08-25T05:01:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.034060664, 2023-08-25T05:01:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000028407 2023-08-25T05:01:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-25T05:01:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 547824 2023-08-25T05:01:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1433, 2023-08-25T05:01:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1410, 2023-08-25T05:01:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1756640, 2023-08-25T05:01:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 522186, 2023-08-25T05:01:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2023-08-25T05:01:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2023-08-25T05:01:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.124893909, 2023-08-25T05:01:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.035933429, 2023-08-25T05:01:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.051209527, 2023-08-25T05:01:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.037455832, 2023-08-25T05:01:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1410, 2023-08-25T05:01:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.043494584, 2023-08-25T05:01:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030847 2023-08-25T05:01:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1394432, 2023-08-25T05:01:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 186928 2023-08-25T05:01:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2023-08-25T05:01:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2023-08-25T05:01:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2023-08-25T05:01:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2023-08-25T05:01:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5418, 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.124893683, 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000023051 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2023-08-25T05:01:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2023-08-25T05:01:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2023-08-25T05:01:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2023-08-25T05:01:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2023-08-25T05:01:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2023-08-25T05:01:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2023-08-25T05:01:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2023-08-25T05:01:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2023-08-25T05:01:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2023-08-25T05:01:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2023-08-25T05:01:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2023-08-25T05:01:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 746, 2023-08-25T05:01:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2023-08-25T05:01:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2023-08-25T05:01:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 746, 2023-08-25T05:01:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 746, 2023-08-25T05:01:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 38, 2023-08-25T05:01:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2023-08-25T05:01:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2023-08-25T05:01:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 34, 2023-08-25T05:01:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2023-08-25T05:01:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2023-08-25T05:01:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2023-08-25T05:01:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2023-08-25T05:01:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2023-08-25T05:01:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2023-08-25T05:01:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2023-08-25T05:01:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2023-08-25T05:01:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2023-08-25T05:01:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2023-08-25T05:01:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2023-08-25T05:01:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2023-08-25T05:01:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2023-08-25T05:01:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2023-08-25T05:01:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2023-08-25T05:01:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2023-08-25T05:01:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2023-08-25T05:01:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2023-08-25T05:01:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2023-08-25T05:01:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2023-08-25T05:01:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2023-08-25T05:01:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:01:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:01:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2023-08-25T05:01:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T05:01:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T05:01:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2023-08-25T05:01:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2023-08-25T05:01:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2023-08-25T05:01:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2023-08-25T05:01:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:01:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:01:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T05:01:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2023-08-25T05:01:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2023-08-25T05:01:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2023-08-25T05:01:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2023-08-25T05:01:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2023-08-25T05:01:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2023-08-25T05:01:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2023-08-25T05:01:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2023-08-25T05:01:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2023-08-25T05:01:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2023-08-25T05:01:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2023-08-25T05:01:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2023-08-25T05:01:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:01:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2023-08-25T05:01:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2023-08-25T05:01:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2023-08-25T05:01:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2023-08-25T05:01:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2023-08-25T05:01:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:01:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:01:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2023-08-25T05:01:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2023-08-25T05:01:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T05:01:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2023-08-25T05:01:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2023-08-25T05:01:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2023-08-25T05:01:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2023-08-25T05:01:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2023-08-25T05:01:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2023-08-25T05:01:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2023-08-25T05:01:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2023-08-25T05:01:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2023-08-25T05:01:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2023-08-25T05:01:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1944, 2023-08-25T05:01:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2023-08-25T05:01:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2023-08-25T05:01:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2023-08-25T05:01:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2023-08-25T05:01:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2023-08-25T05:01:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 3, 2023-08-25T05:01:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2023-08-25T05:01:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2023-08-25T05:01:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2023-08-25T05:01:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2023-08-25T05:01:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 8020996, 2023-08-25T05:01:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 8020422, 2023-08-25T05:01:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2023-08-25T05:01:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 1944 2023-08-25T05:01:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2023-08-25T05:01:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 720, 2023-08-25T05:01:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1192, 2023-08-25T05:01:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 472, 2023-08-25T05:01:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 39, 2023-08-25T05:01:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 66, 2023-08-25T05:01:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 27, 2023-08-25T05:01:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 746, 2023-08-25T05:01:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1340, 2023-08-25T05:01:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 594, 2023-08-25T05:01:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2023-08-25T05:01:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2023-08-25T05:01:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2023-08-25T05:01:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 50292, 2023-08-25T05:01:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 207152 2023-08-25T05:01:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2023-08-25T05:01:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2023-08-25T05:01:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2023-08-25T05:01:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2023-08-25T05:01:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2023-08-25T05:01:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2023-08-25T05:01:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2023-08-25T05:01:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2023-08-25T05:01:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2023-08-25T05:01:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2023-08-25T05:01:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2023-08-25T05:01:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.340 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.340 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.341 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2023-08-25T05:01:16.341 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.341 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.342 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.342 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2023-08-25T05:01:16.342 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.342 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.343 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.343 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2023-08-25T05:01:16.343 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.344 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.344 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.345 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2023-08-25T05:01:16.345 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.345 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.346 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.346 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2023-08-25T05:01:16.347 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.347 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.347 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.348 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2023-08-25T05:01:16.348 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.348 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.348 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.349 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2023-08-25T05:01:16.349 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.349 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.349 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.350 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2023-08-25T05:01:16.350 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.351 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.351 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.351 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2023-08-25T05:01:16.351 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.352 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.352 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.352 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2023-08-25T05:01:16.352 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.353 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.353 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.353 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2023-08-25T05:01:16.353 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.354 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.354 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.354 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2023-08-25T05:01:16.354 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.355 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.355 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.356 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2023-08-25T05:01:16.356 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.356 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.356 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.358 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2023-08-25T05:01:16.358 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.358 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.358 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.359 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2023-08-25T05:01:16.359 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.359 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.360 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.360 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2023-08-25T05:01:16.360 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.361 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.362 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.362 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2023-08-25T05:01:16.363 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.363 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.363 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.364 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2023-08-25T05:01:16.364 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.364 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.364 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.365 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2023-08-25T05:01:16.365 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.365 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.366 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2023-08-25T05:01:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.367 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2023-08-25T05:01:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.368 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2023-08-25T05:01:16.369 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.369 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.369 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.370 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:01:16.370 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:01:16.370 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2023-08-25T05:01:16.371 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.371 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.371 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.372 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2023-08-25T05:01:16.372 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2023-08-25T05:01:16.372 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2023-08-25T05:01:16.372 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3923 2023-08-25T05:01:16.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.374 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2023-08-25T05:01:16.374 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:16.374 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:16.374 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:16.375 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:16.375 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:16.375 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:16.375 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:16.376 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:16.376 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:16.376 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:16.377 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:16.377 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:16.377 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:16.378 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:16.378 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:16.378 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:16.378 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:16.379 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:16.379 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:16.379 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:16.380 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:16.380 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:16.380 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:16.381 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:16.381 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:16.381 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:16.382 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2023-08-25T05:01:16.383 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:16.383 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:16.383 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:16.383 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:16.384 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 218758, 2023-08-25T05:01:16.384 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 826, 2023-08-25T05:01:16.384 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:16.385 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:16.385 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2023-08-25T05:01:16.386 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2023-08-25T05:01:16.386 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:16.386 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:16.387 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:16.387 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:16.388 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9472, 2023-08-25T05:01:16.388 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 183, 2023-08-25T05:01:16.389 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:16.389 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:16.389 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:16.390 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2023-08-25T05:01:16.391 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 2283316, 2023-08-25T05:01:16.392 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 24959, 2023-08-25T05:01:16.392 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:16.393 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2023-08-25T05:01:16.393 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:16.393 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2023-08-25T05:01:16.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.394 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2023-08-25T05:01:16.394 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2023-08-25T05:01:16.395 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2023-08-25T05:01:16.395 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3207, 2023-08-25T05:01:16.396 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2023-08-25T05:01:16.396 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2023-08-25T05:01:16.397 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3207, 2023-08-25T05:01:16.397 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2023-08-25T05:01:16.397 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3207, 2023-08-25T05:01:16.397 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.703955083, 2023-08-25T05:01:16.398 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000531323 2023-08-25T05:01:16.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.398 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2023-08-25T05:01:16.398 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2023-08-25T05:01:16.399 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3207, 2023-08-25T05:01:16.399 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3208 2023-08-25T05:01:16.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.399 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3207, 2023-08-25T05:01:16.399 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3207, 2023-08-25T05:01:16.400 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2023-08-25T05:01:16.400 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2023-08-25T05:01:16.400 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2023-08-25T05:01:16.400 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2048, 2023-08-25T05:01:16.401 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2023-08-25T05:01:16.401 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1157, 2023-08-25T05:01:16.402 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2023-08-25T05:01:16.402 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2023-08-25T05:01:16.403 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2023-08-25T05:01:16.403 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2023-08-25T05:01:16.404 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2023-08-25T05:01:16.404 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2023-08-25T05:01:16.404 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2023-08-25T05:01:16.404 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2023-08-25T05:01:16.405 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2023-08-25T05:01:16.405 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2023-08-25T05:01:16.405 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2023-08-25T05:01:16.406 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2023-08-25T05:01:16.406 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:01:16.406 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2023-08-25T05:01:16.407 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:01:16.407 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2023-08-25T05:01:16.407 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2023-08-25T05:01:16.407 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2023-08-25T05:01:16.408 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:01:16.408 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2023-08-25T05:01:16.408 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:01:16.409 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2023-08-25T05:01:16.409 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2023-08-25T05:01:16.409 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2023-08-25T05:01:16.409 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2023-08-25T05:01:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2023-08-25T05:01:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2023-08-25T05:01:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2023-08-25T05:01:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2023-08-25T05:01:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2023-08-25T05:01:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2023-08-25T05:01:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2023-08-25T05:01:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2023-08-25T05:01:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2023-08-25T05:01:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2023-08-25T05:01:16.412 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2023-08-25T05:01:16.412 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2023-08-25T05:01:16.412 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2023-08-25T05:01:16.412 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 110, 2023-08-25T05:01:16.412 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2023-08-25T05:01:16.413 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 23, 2023-08-25T05:01:16.413 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2023-08-25T05:01:16.413 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2023-08-25T05:01:16.414 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2023-08-25T05:01:16.414 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2023-08-25T05:01:16.414 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2023-08-25T05:01:16.414 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2023-08-25T05:01:16.414 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2023-08-25T05:01:16.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.415 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2023-08-25T05:01:16.415 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2023-08-25T05:01:16.416 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:01:16.416 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2023-08-25T05:01:16.416 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2023-08-25T05:01:16.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.417 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2023-08-25T05:01:16.417 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2023-08-25T05:01:16.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.418 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2023-08-25T05:01:16.418 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2023-08-25T05:01:16.418 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2023-08-25T05:01:16.418 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2023-08-25T05:01:16.419 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2023-08-25T05:01:16.419 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2023-08-25T05:01:16.419 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2023-08-25T05:01:16.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.420 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:16.420 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:16.420 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:16.421 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:16.421 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3560, 2023-08-25T05:01:16.421 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5783936, 2023-08-25T05:01:16.421 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:16.421 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3560, 2023-08-25T05:01:16.422 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:16.422 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:16.422 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3560, 2023-08-25T05:01:16.422 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5783936, 2023-08-25T05:01:16.422 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:16.423 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.423 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.423 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.423 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.425 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2023-08-25T05:01:16.425 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:16.426 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:16.426 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:16.426 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:16.427 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:16.427 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:16.428 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:16.429 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3207, 2023-08-25T05:01:16.429 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3826697, 2023-08-25T05:01:16.429 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3207, 2023-08-25T05:01:16.429 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3826697, 2023-08-25T05:01:16.429 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:16.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.430 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.430 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.431 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2023-08-25T05:01:16.431 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:16.431 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2023-08-25T05:01:16.431 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:16.431 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:16.432 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:16.432 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:16.432 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:16.432 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3207, 2023-08-25T05:01:16.433 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3207, 2023-08-25T05:01:16.433 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3207, 2023-08-25T05:01:16.433 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3207, 2023-08-25T05:01:16.433 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:16.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.434 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.434 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:01:16.435 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:16.435 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:16.435 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:16.435 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:16.435 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:16.436 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:16.436 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:16.436 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:16.436 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:16.436 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:16.438 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x560fa3c98960": { 2023-08-25T05:01:16.438 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:16.438 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:16.438 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:16.439 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:16.439 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:16.439 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:16.439 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:16.439 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:16.440 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:16.440 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2023-08-25T05:01:16.440 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2023-08-25T05:01:16.440 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:16.440 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:16.441 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:16.441 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:16.441 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.441 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:16.441 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-08-25T05:01:16.442 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:15 smithi145 ceph-mon[97681]: pgmap v507: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:16.442 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:15 smithi145 ceph-mon[97681]: osdmap e110: 12 total, 12 up, 12 in 2023-08-25T05:01:16.616 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-08-25T05:01:16.617 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-08-25T05:01:16.617 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-08-25T05:01:16.617 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-08-25T05:01:16.618 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-08-25T05:01:16.618 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-08-25T05:01:16.618 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T05:01:16.618 DEBUG:tasks.check_counter:Getting stats from i 2023-08-25T05:01:16.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.i perf dump 2023-08-25T05:01:17.187 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T05:01:18.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:17 smithi162 ceph-mon[130941]: pgmap v509: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:18.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:17 smithi183 ceph-mon[131414]: pgmap v509: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:18.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:17 smithi145 ceph-mon[97681]: pgmap v509: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:18.429 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-08-25T05:01:18.430 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:18.430 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2437, 2023-08-25T05:01:18.430 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3045, 2023-08-25T05:01:18.431 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1997069, 2023-08-25T05:01:18.431 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 4665173, 2023-08-25T05:01:18.431 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 12, 2023-08-25T05:01:18.431 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2023-08-25T05:01:18.432 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.149849226, 2023-08-25T05:01:18.432 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.055394081, 2023-08-25T05:01:18.432 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.079480026, 2023-08-25T05:01:18.432 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.022022753, 2023-08-25T05:01:18.433 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:18.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3044, 2023-08-25T05:01:18.433 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.128416732, 2023-08-25T05:01:18.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042186 2023-08-25T05:01:18.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.434 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:18.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3223, 2023-08-25T05:01:18.435 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000446986, 2023-08-25T05:01:18.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000138 2023-08-25T05:01:18.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.436 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1402576, 2023-08-25T05:01:18.436 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 202000 2023-08-25T05:01:18.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.437 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:18.438 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 148, 2023-08-25T05:01:18.438 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 376, 2023-08-25T05:01:18.439 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 51823, 2023-08-25T05:01:18.439 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 966036, 2023-08-25T05:01:18.439 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 19, 2023-08-25T05:01:18.439 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2023-08-25T05:01:18.440 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.023545084, 2023-08-25T05:01:18.440 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.009566388, 2023-08-25T05:01:18.440 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.007906598, 2023-08-25T05:01:18.440 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.003347766, 2023-08-25T05:01:18.441 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:18.441 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 373, 2023-08-25T05:01:18.441 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.029904647, 2023-08-25T05:01:18.441 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000080173 2023-08-25T05:01:18.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.442 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:18.442 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 13, 2023-08-25T05:01:18.443 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000001396, 2023-08-25T05:01:18.443 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000107 2023-08-25T05:01:18.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.443 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1392, 2023-08-25T05:01:18.444 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 642912 2023-08-25T05:01:18.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.444 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:18.444 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5658510, 2023-08-25T05:01:18.444 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 5659948, 2023-08-25T05:01:18.445 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 957223863, 2023-08-25T05:01:18.445 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 896437725, 2023-08-25T05:01:18.445 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2023-08-25T05:01:18.445 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2023-08-25T05:01:18.446 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 134.208527024, 2023-08-25T05:01:18.446 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 51.364954509, 2023-08-25T05:01:18.446 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 92.512305801, 2023-08-25T05:01:18.446 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.005995586, 2023-08-25T05:01:18.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:18.447 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5659947, 2023-08-25T05:01:18.448 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 43.020386042, 2023-08-25T05:01:18.448 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000007600 2023-08-25T05:01:18.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:18.449 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11314537, 2023-08-25T05:01:18.450 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.896984704, 2023-08-25T05:01:18.450 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000079 2023-08-25T05:01:18.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-25T05:01:18.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2023-08-25T05:01:18.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.452 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2023-08-25T05:01:18.452 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2023-08-25T05:01:18.452 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2023-08-25T05:01:18.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.453 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2023-08-25T05:01:18.453 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:18.454 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:18.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1282, 2023-08-25T05:01:18.454 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.620152056, 2023-08-25T05:01:18.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000483737 2023-08-25T05:01:18.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.455 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2023-08-25T05:01:18.456 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2023-08-25T05:01:18.456 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2023-08-25T05:01:18.456 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 68, 2023-08-25T05:01:18.456 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002194739, 2023-08-25T05:01:18.456 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000032275 2023-08-25T05:01:18.457 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.457 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2023-08-25T05:01:18.457 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 539, 2023-08-25T05:01:18.458 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 424, 2023-08-25T05:01:18.458 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2023-08-25T05:01:18.458 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 424, 2023-08-25T05:01:18.459 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.543377881, 2023-08-25T05:01:18.460 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005998532 2023-08-25T05:01:18.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.461 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2023-08-25T05:01:18.461 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 115, 2023-08-25T05:01:18.461 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 1, 2023-08-25T05:01:18.462 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2023-08-25T05:01:18.462 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 51, 2023-08-25T05:01:18.463 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 5, 2023-08-25T05:01:18.463 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2023-08-25T05:01:18.463 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 717, 2023-08-25T05:01:18.464 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 171, 2023-08-25T05:01:18.464 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 511, 2023-08-25T05:01:18.465 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 35, 2023-08-25T05:01:18.465 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 473, 2023-08-25T05:01:18.466 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 644, 2023-08-25T05:01:18.466 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 7, 2023-08-25T05:01:18.467 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 7, 2023-08-25T05:01:18.468 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 44, 2023-08-25T05:01:18.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 3146, 2023-08-25T05:01:18.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 2241, 2023-08-25T05:01:18.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 115, 2023-08-25T05:01:18.469 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 37, 2023-08-25T05:01:18.469 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2023-08-25T05:01:18.470 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2023-08-25T05:01:18.470 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 587, 2023-08-25T05:01:18.470 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2023-08-25T05:01:18.470 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2023-08-25T05:01:18.470 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 21, 2023-08-25T05:01:18.471 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 565, 2023-08-25T05:01:18.471 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 18, 2023-08-25T05:01:18.471 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 589, 2023-08-25T05:01:18.471 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2023-08-25T05:01:18.472 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 150, 2023-08-25T05:01:18.472 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2023-08-25T05:01:18.472 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 991, 2023-08-25T05:01:18.473 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 6101576512, 2023-08-25T05:01:18.474 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 3, 2023-08-25T05:01:18.474 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 150, 2023-08-25T05:01:18.474 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2023-08-25T05:01:18.474 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2023-08-25T05:01:18.475 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2023-08-25T05:01:18.475 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2023-08-25T05:01:18.476 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2023-08-25T05:01:18.476 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2023-08-25T05:01:18.477 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 150, 2023-08-25T05:01:18.479 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 20, 2023-08-25T05:01:18.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 515, 2023-08-25T05:01:18.480 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 902, 2023-08-25T05:01:18.480 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2023-08-25T05:01:18.480 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-25T05:01:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-25T05:01:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 972, 2023-08-25T05:01:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 181, 2023-08-25T05:01:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 30, 2023-08-25T05:01:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 174 2023-08-25T05:01:18.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.482 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2023-08-25T05:01:18.482 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 24, 2023-08-25T05:01:18.483 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 174, 2023-08-25T05:01:18.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 99, 2023-08-25T05:01:18.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 148, 2023-08-25T05:01:18.485 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 54, 2023-08-25T05:01:18.485 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 19, 2023-08-25T05:01:18.486 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2023-08-25T05:01:18.486 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2023-08-25T05:01:18.487 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 48, 2023-08-25T05:01:18.487 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 12, 2023-08-25T05:01:18.488 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 117, 2023-08-25T05:01:18.488 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2023-08-25T05:01:18.489 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2023-08-25T05:01:18.489 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2023-08-25T05:01:18.490 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2023-08-25T05:01:18.490 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2023-08-25T05:01:18.490 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2023-08-25T05:01:18.491 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2023-08-25T05:01:18.491 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 21, 2023-08-25T05:01:18.491 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2023-08-25T05:01:18.491 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 5, 2023-08-25T05:01:18.492 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2023-08-25T05:01:18.492 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2023-08-25T05:01:18.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.492 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2023-08-25T05:01:18.493 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2023-08-25T05:01:18.493 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 1944, 2023-08-25T05:01:18.493 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2023-08-25T05:01:18.493 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2023-08-25T05:01:18.494 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1944, 2023-08-25T05:01:18.494 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2023-08-25T05:01:18.494 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2023-08-25T05:01:18.494 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 3, 2023-08-25T05:01:18.494 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2023-08-25T05:01:18.495 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2023-08-25T05:01:18.495 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 3, 2023-08-25T05:01:18.496 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2023-08-25T05:01:18.496 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2023-08-25T05:01:18.496 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2023-08-25T05:01:18.496 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2023-08-25T05:01:18.497 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 8020996, 2023-08-25T05:01:18.497 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2023-08-25T05:01:18.497 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2023-08-25T05:01:18.498 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 626, 2023-08-25T05:01:18.499 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.132330977, 2023-08-25T05:01:18.500 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003406279 2023-08-25T05:01:18.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.502 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2023-08-25T05:01:18.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.503 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2023-08-25T05:01:18.503 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 702, 2023-08-25T05:01:18.504 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1230, 2023-08-25T05:01:18.504 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 528, 2023-08-25T05:01:18.505 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 124, 2023-08-25T05:01:18.505 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 195, 2023-08-25T05:01:18.506 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 71, 2023-08-25T05:01:18.507 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 717, 2023-08-25T05:01:18.507 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1392, 2023-08-25T05:01:18.508 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 675, 2023-08-25T05:01:18.508 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 7, 2023-08-25T05:01:18.509 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 740, 2023-08-25T05:01:18.509 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 733, 2023-08-25T05:01:18.510 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 61696, 2023-08-25T05:01:18.511 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 198960 2023-08-25T05:01:18.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.514 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2023-08-25T05:01:18.515 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 2136, 2023-08-25T05:01:18.515 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 136, 2023-08-25T05:01:18.515 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 539, 2023-08-25T05:01:18.517 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 93, 2023-08-25T05:01:18.518 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 300, 2023-08-25T05:01:18.518 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2023-08-25T05:01:18.519 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2023-08-25T05:01:18.519 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.018039873, 2023-08-25T05:01:18.519 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001127492 2023-08-25T05:01:18.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.520 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2023-08-25T05:01:18.520 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 25, 2023-08-25T05:01:18.521 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.023250614, 2023-08-25T05:01:18.521 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000930024 2023-08-25T05:01:18.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.522 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2023-08-25T05:01:18.522 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.522 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.522 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.523 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2023-08-25T05:01:18.523 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 28, 2023-08-25T05:01:18.523 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.165890661, 2023-08-25T05:01:18.524 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005924666 2023-08-25T05:01:18.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.524 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2023-08-25T05:01:18.525 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 130, 2023-08-25T05:01:18.525 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.171787199, 2023-08-25T05:01:18.525 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001321439 2023-08-25T05:01:18.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.526 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2023-08-25T05:01:18.526 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.526 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.527 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.528 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2023-08-25T05:01:18.528 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.529 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.529 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.530 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2023-08-25T05:01:18.531 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.531 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.532 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.533 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2023-08-25T05:01:18.533 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.534 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.534 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.536 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2023-08-25T05:01:18.537 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.537 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.537 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.538 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2023-08-25T05:01:18.539 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.539 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.540 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.542 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2023-08-25T05:01:18.543 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.544 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.544 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.545 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2023-08-25T05:01:18.546 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.546 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.547 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.547 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2023-08-25T05:01:18.548 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.549 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.549 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.550 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2023-08-25T05:01:18.551 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2023-08-25T05:01:18.551 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021172176, 2023-08-25T05:01:18.551 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001114325 2023-08-25T05:01:18.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.552 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2023-08-25T05:01:18.553 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 37, 2023-08-25T05:01:18.553 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.031681855, 2023-08-25T05:01:18.553 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000856266 2023-08-25T05:01:18.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.554 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2023-08-25T05:01:18.554 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 129, 2023-08-25T05:01:18.554 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.985725341, 2023-08-25T05:01:18.554 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.015393219 2023-08-25T05:01:18.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.555 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2023-08-25T05:01:18.555 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.556 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.556 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.556 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2023-08-25T05:01:18.556 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.557 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.557 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2023-08-25T05:01:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.559 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2023-08-25T05:01:18.560 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.560 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.560 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.561 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2023-08-25T05:01:18.561 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.562 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.562 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.563 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2023-08-25T05:01:18.563 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.563 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.564 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.564 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2023-08-25T05:01:18.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.565 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.566 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2023-08-25T05:01:18.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.567 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.568 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2023-08-25T05:01:18.569 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.569 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.569 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.570 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2023-08-25T05:01:18.571 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.572 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.572 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.573 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2023-08-25T05:01:18.573 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.573 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.574 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.574 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2023-08-25T05:01:18.574 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 40, 2023-08-25T05:01:18.575 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.125830162, 2023-08-25T05:01:18.575 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003145754 2023-08-25T05:01:18.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.575 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2023-08-25T05:01:18.575 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2023-08-25T05:01:18.576 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2023-08-25T05:01:18.576 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.576 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.576 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.577 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2023-08-25T05:01:18.577 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2023-08-25T05:01:18.577 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2023-08-25T05:01:18.577 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2023-08-25T05:01:18.578 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2023-08-25T05:01:18.578 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2023-08-25T05:01:18.578 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 249, 2023-08-25T05:01:18.578 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 124, 2023-08-25T05:01:18.579 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 938 2023-08-25T05:01:18.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.579 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2023-08-25T05:01:18.579 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:18.579 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:18.580 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:18.580 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:18.580 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:18.580 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:18.580 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:18.581 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:18.581 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:18.581 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:18.581 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:18.582 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:18.582 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:18.582 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:18.582 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:18.582 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:18.583 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:18.583 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:18.583 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:18.583 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:18.583 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:18.584 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:18.584 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:18.584 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:18.584 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:18.586 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:18.586 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2023-08-25T05:01:18.586 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:18.586 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:18.586 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:18.587 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:18.587 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 462159, 2023-08-25T05:01:18.587 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 483, 2023-08-25T05:01:18.587 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 88, 2023-08-25T05:01:18.587 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 1, 2023-08-25T05:01:18.588 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2023-08-25T05:01:18.588 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2023-08-25T05:01:18.588 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:18.588 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:18.589 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:18.589 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:18.589 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9472, 2023-08-25T05:01:18.589 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 183, 2023-08-25T05:01:18.589 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:18.590 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:18.590 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:18.590 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2023-08-25T05:01:18.590 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 2318479, 2023-08-25T05:01:18.590 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 23239, 2023-08-25T05:01:18.591 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:18.591 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2023-08-25T05:01:18.591 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:18.591 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2023-08-25T05:01:18.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.592 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2023-08-25T05:01:18.592 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2023-08-25T05:01:18.592 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2023-08-25T05:01:18.593 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 950, 2023-08-25T05:01:18.593 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 4271198, 2023-08-25T05:01:18.593 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 27, 2023-08-25T05:01:18.593 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 950, 2023-08-25T05:01:18.593 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2023-08-25T05:01:18.594 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 950, 2023-08-25T05:01:18.594 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.503933241, 2023-08-25T05:01:18.594 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002635719 2023-08-25T05:01:18.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.594 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2023-08-25T05:01:18.595 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2023-08-25T05:01:18.595 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 950, 2023-08-25T05:01:18.595 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1050 2023-08-25T05:01:18.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.595 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 950, 2023-08-25T05:01:18.596 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 161, 2023-08-25T05:01:18.596 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 789, 2023-08-25T05:01:18.596 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2023-08-25T05:01:18.596 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2023-08-25T05:01:18.597 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2023-08-25T05:01:18.597 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2023-08-25T05:01:18.597 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 8, 2023-08-25T05:01:18.597 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 634, 2023-08-25T05:01:18.598 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 27, 2023-08-25T05:01:18.598 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2023-08-25T05:01:18.598 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2023-08-25T05:01:18.598 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2023-08-25T05:01:18.599 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2023-08-25T05:01:18.599 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 41, 2023-08-25T05:01:18.599 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2023-08-25T05:01:18.599 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2023-08-25T05:01:18.599 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2023-08-25T05:01:18.600 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 150, 2023-08-25T05:01:18.600 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2023-08-25T05:01:18.600 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2023-08-25T05:01:18.600 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2023-08-25T05:01:18.600 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2023-08-25T05:01:18.601 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2023-08-25T05:01:18.601 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2023-08-25T05:01:18.601 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2023-08-25T05:01:18.601 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2023-08-25T05:01:18.601 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2023-08-25T05:01:18.602 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2023-08-25T05:01:18.602 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2023-08-25T05:01:18.602 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2023-08-25T05:01:18.602 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2023-08-25T05:01:18.602 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2023-08-25T05:01:18.603 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2023-08-25T05:01:18.603 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2023-08-25T05:01:18.603 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2023-08-25T05:01:18.603 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2023-08-25T05:01:18.603 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2023-08-25T05:01:18.604 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2023-08-25T05:01:18.604 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2023-08-25T05:01:18.604 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2023-08-25T05:01:18.604 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2023-08-25T05:01:18.605 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2023-08-25T05:01:18.605 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2023-08-25T05:01:18.605 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2023-08-25T05:01:18.605 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2023-08-25T05:01:18.605 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 110, 2023-08-25T05:01:18.606 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2023-08-25T05:01:18.606 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 23, 2023-08-25T05:01:18.606 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2023-08-25T05:01:18.606 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2023-08-25T05:01:18.606 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2023-08-25T05:01:18.607 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2023-08-25T05:01:18.607 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 138, 2023-08-25T05:01:18.607 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 4, 2023-08-25T05:01:18.607 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 18 2023-08-25T05:01:18.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.608 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2023-08-25T05:01:18.608 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2023-08-25T05:01:18.608 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2023-08-25T05:01:18.608 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2023-08-25T05:01:18.608 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2023-08-25T05:01:18.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.609 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2023-08-25T05:01:18.609 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 5 2023-08-25T05:01:18.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.609 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2023-08-25T05:01:18.610 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2023-08-25T05:01:18.610 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 10, 2023-08-25T05:01:18.610 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2023-08-25T05:01:18.611 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 5, 2023-08-25T05:01:18.612 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 12, 2023-08-25T05:01:18.612 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2023-08-25T05:01:18.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.612 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:18.613 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:18.613 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:18.613 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:18.613 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 5661093, 2023-08-25T05:01:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 449776463, 2023-08-25T05:01:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 5661093, 2023-08-25T05:01:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:18.615 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5661093, 2023-08-25T05:01:18.615 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 449776463, 2023-08-25T05:01:18.615 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:18.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.616 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.617 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2023-08-25T05:01:18.617 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:18.618 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:18.618 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:18.618 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:18.618 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:18.619 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:18.619 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:18.619 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 950, 2023-08-25T05:01:18.619 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4272494, 2023-08-25T05:01:18.620 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 950, 2023-08-25T05:01:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4272494, 2023-08-25T05:01:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2023-08-25T05:01:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2023-08-25T05:01:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2023-08-25T05:01:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2023-08-25T05:01:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2023-08-25T05:01:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 950, 2023-08-25T05:01:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 950, 2023-08-25T05:01:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 950, 2023-08-25T05:01:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 950, 2023-08-25T05:01:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2023-08-25T05:01:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 12, 2023-08-25T05:01:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1356, 2023-08-25T05:01:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 12, 2023-08-25T05:01:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 8, 2023-08-25T05:01:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1356, 2023-08-25T05:01:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55b2b4e5a6e0": { 2023-08-25T05:01:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2023-08-25T05:01:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1944, 2023-08-25T05:01:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 3826692, 2023-08-25T05:01:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1944, 2023-08-25T05:01:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 626, 2023-08-25T05:01:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3826692, 2023-08-25T05:01:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:18.650 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-08-25T05:01:18.830 INFO:tasks.check_counter:Daemon mds.i mds.exported=21 2023-08-25T05:01:18.831 INFO:tasks.check_counter:Daemon mds.i mds.imported=18 2023-08-25T05:01:18.832 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=24 2023-08-25T05:01:18.832 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=174 2023-08-25T05:01:18.832 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=3 2023-08-25T05:01:18.833 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-08-25T05:01:18.833 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-08-25T05:01:18.834 DEBUG:tasks.check_counter:Getting stats from l 2023-08-25T05:01:18.834 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ba6b39e4f4f77fb95c747099b53db0467d435f56 shell --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c -- ceph daemon mds.l perf dump 2023-08-25T05:01:19.414 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/mon.c/config 2023-08-25T05:01:19.915 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:19 smithi183 ceph-mon[131414]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:20.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:19 smithi162 ceph-mon[130941]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:20.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:19 smithi145 ceph-mon[97681]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-08-25T05:01:20.689 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-08-25T05:01:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2023-08-25T05:01:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2023-08-25T05:01:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2023-08-25T05:01:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2023-08-25T05:01:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2023-08-25T05:01:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2023-08-25T05:01:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2023-08-25T05:01:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.001277588, 2023-08-25T05:01:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2023-08-25T05:01:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000305438, 2023-08-25T05:01:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-25T05:01:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2023-08-25T05:01:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2023-08-25T05:01:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 319, 2023-08-25T05:01:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 308, 2023-08-25T05:01:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1306592, 2023-08-25T05:01:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 184544, 2023-08-25T05:01:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2023-08-25T05:01:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2023-08-25T05:01:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.042740072, 2023-08-25T05:01:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.012791649, 2023-08-25T05:01:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.030057421, 2023-08-25T05:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006520660, 2023-08-25T05:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 308, 2023-08-25T05:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.009458544, 2023-08-25T05:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030709 2023-08-25T05:01:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1306592, 2023-08-25T05:01:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 184544 2023-08-25T05:01:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2023-08-25T05:01:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3, 2023-08-25T05:01:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 235, 2023-08-25T05:01:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 480, 2023-08-25T05:01:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 270272, 2023-08-25T05:01:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2023-08-25T05:01:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2023-08-25T05:01:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.010636254, 2023-08-25T05:01:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.006717554, 2023-08-25T05:01:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001669150, 2023-08-25T05:01:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-25T05:01:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2023-08-25T05:01:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 232, 2023-08-25T05:01:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013744250, 2023-08-25T05:01:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000059242 2023-08-25T05:01:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2023-08-25T05:01:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 480, 2023-08-25T05:01:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 270272 2023-08-25T05:01:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2023-08-25T05:01:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2023-08-25T05:01:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2023-08-25T05:01:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2023-08-25T05:01:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2023-08-25T05:01:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2023-08-25T05:01:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2023-08-25T05:01:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2023-08-25T05:01:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2023-08-25T05:01:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2023-08-25T05:01:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-25T05:01:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2023-08-25T05:01:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-25T05:01:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2023-08-25T05:01:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2023-08-25T05:01:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2023-08-25T05:01:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-25T05:01:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2023-08-25T05:01:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2023-08-25T05:01:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2023-08-25T05:01:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2023-08-25T05:01:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2023-08-25T05:01:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-25T05:01:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2023-08-25T05:01:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-25T05:01:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2023-08-25T05:01:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2023-08-25T05:01:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2023-08-25T05:01:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2023-08-25T05:01:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2023-08-25T05:01:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-25T05:01:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2023-08-25T05:01:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2023-08-25T05:01:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2023-08-25T05:01:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2023-08-25T05:01:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2023-08-25T05:01:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2023-08-25T05:01:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2023-08-25T05:01:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2023-08-25T05:01:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2023-08-25T05:01:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 148754, 2023-08-25T05:01:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 31, 2023-08-25T05:01:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2023-08-25T05:01:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2023-08-25T05:01:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2023-08-25T05:01:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2023-08-25T05:01:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2023-08-25T05:01:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2023-08-25T05:01:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2023-08-25T05:01:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2023-08-25T05:01:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2023-08-25T05:01:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2023-08-25T05:01:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2023-08-25T05:01:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2023-08-25T05:01:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2023-08-25T05:01:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2023-08-25T05:01:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2023-08-25T05:01:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2023-08-25T05:01:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-25T05:01:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2023-08-25T05:01:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2023-08-25T05:01:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2023-08-25T05:01:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 318, 2023-08-25T05:01:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1259124, 2023-08-25T05:01:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2023-08-25T05:01:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 318, 2023-08-25T05:01:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2023-08-25T05:01:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2023-08-25T05:01:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 318, 2023-08-25T05:01:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1259124, 2023-08-25T05:01:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2023-08-25T05:01:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2023-08-25T05:01:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2023-08-25T05:01:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2023-08-25T05:01:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: } 2023-08-25T05:01:20.722 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-08-25T05:01:21.073 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-08-25T05:01:21.073 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-08-25T05:01:21.074 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-08-25T05:01:21.074 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-08-25T05:01:21.074 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-08-25T05:01:21.074 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-08-25T05:01:21.075 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-08-25T05:01:21.075 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-08-25T05:01:21.085 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-25T05:01:21.086 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T05:01:21.086 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T05:01:21.137 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T05:01:21.138 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T05:01:21.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T05:01:21.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 -- 172.21.15.145:0/3687453316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 msgr2=0x7f0a6c100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3687453316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c100d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009990 tx=0x7f0a5c009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 -- 172.21.15.145:0/3687453316 shutdown_connections 2023-08-25T05:01:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3687453316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c101250 0x7f0a6c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3687453316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c1068a0 0x7f0a6c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.489 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3687453316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c100d10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 -- 172.21.15.145:0/3687453316 >> 172.21.15.145:0/3687453316 conn(0x7f0a6c0fc340 msgr2=0x7f0a6c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:21.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 -- 172.21.15.145:0/3687453316 shutdown_connections 2023-08-25T05:01:21.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.487+0000 7f0a73bff700 1 -- 172.21.15.145:0/3687453316 wait complete. 2023-08-25T05:01:21.491 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 Processor -- start 2023-08-25T05:01:21.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 -- start start 2023-08-25T05:01:21.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:21.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 0x7f0a6c073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:21.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c1068a0 0x7f0a6c0736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:21.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1a9300 con 0x7f0a6c1068a0 2023-08-25T05:01:21.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1a9440 con 0x7f0a6c101250 2023-08-25T05:01:21.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.489+0000 7f0a73bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1a9580 con 0x7f0a6c100890 2023-08-25T05:01:21.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:21.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:47658/0 (socket says 172.21.15.145:47658) 2023-08-25T05:01:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 -- 172.21.15.145:0/3443844645 learned_addr learned my addr 172.21.15.145:0/3443844645 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7119a700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 0x7f0a6c073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7219c700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c1068a0 0x7f0a6c0736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 -- 172.21.15.145:0/3443844645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 msgr2=0x7f0a6c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:21.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 0x7f0a6c073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 -- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c1068a0 msgr2=0x7f0a6c0736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:21.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c1068a0 0x7f0a6c0736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 -- 172.21.15.145:0/3443844645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0095f0 con 0x7f0a6c100890 2023-08-25T05:01:21.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7119a700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 0x7f0a6c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:01:21.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a7199b700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d980 tx=0x7f0a6800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:21.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.490+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800bb60 con 0x7f0a6c100890 2023-08-25T05:01:21.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.491+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68007640 con 0x7f0a6c100890 2023-08-25T05:01:21.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.491+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c1a9860 con 0x7f0a6c100890 2023-08-25T05:01:21.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.491+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004e90 con 0x7f0a6c100890 2023-08-25T05:01:21.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.491+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1a9dc0 con 0x7f0a6c100890 2023-08-25T05:01:21.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.492+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c067710 con 0x7f0a6c100890 2023-08-25T05:01:21.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.494+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f0a6800bcc0 con 0x7f0a6c100890 2023-08-25T05:01:21.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.496+0000 7f0a62ffd700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 0x7f0a580a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:21.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.496+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f0a680cb780 con 0x7f0a6c100890 2023-08-25T05:01:21.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.496+0000 7f0a7119a700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 0x7f0a580a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:21.498 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.497+0000 7f0a7119a700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 0x7f0a580a2640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00b2e0 tx=0x7f0a5c005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:21.500 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.499+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f0a68094e30 con 0x7f0a6c100890 2023-08-25T05:01:21.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.757+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a6c110900 con 0x7f0a6c100890 2023-08-25T05:01:21.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.759+0000 7f0a62ffd700 1 -- 172.21.15.145:0/3443844645 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+25016 (secure 0 0 0) 0x7f0a6800de60 con 0x7f0a6c100890 2023-08-25T05:01:21.760 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:01:21.761 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":110,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T05:01:14.845060+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4132549840":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4072524950":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6800/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/3264758001":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/459521375":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T05:01:21.763 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.761+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 msgr2=0x7f0a580a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:21.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.761+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 0x7f0a580a2640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00b2e0 tx=0x7f0a5c005fd0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 msgr2=0x7f0a6c072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:21.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d980 tx=0x7f0a6800dd40 comp rx=0 tx=0).stop 2023-08-25T05:01:21.764 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 shutdown_connections 2023-08-25T05:01:21.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a6c1068a0 0x7f0a6c0736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a6c100890 0x7f0a6c072c20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f0a580a0180 0x7f0a580a2640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 --2- 172.21.15.145:0/3443844645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a6c101250 0x7f0a6c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:21.765 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 >> 172.21.15.145:0/3443844645 conn(0x7f0a6c0fc340 msgr2=0x7f0a6c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:21.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 shutdown_connections 2023-08-25T05:01:21.766 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:21.762+0000 7f0a73bff700 1 -- 172.21.15.145:0/3443844645 wait complete. 2023-08-25T05:01:21.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T04:38:48.217192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T04:42:48.249947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T04:42:49.283219+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-25T04:42:52.807279+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-25T05:01:21.787 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T05:01:22.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 -- 172.21.15.145:0/948605502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00073670 msgr2=0x7fbe0010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 --2- 172.21.15.145:0/948605502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00073670 0x7fbe0010cb70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009990 tx=0x7fbdf4009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 -- 172.21.15.145:0/948605502 shutdown_connections 2023-08-25T05:01:22.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 --2- 172.21.15.145:0/948605502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe0010d0b0 0x7fbe0010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 --2- 172.21.15.145:0/948605502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe00072d50 0x7fbe00073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 --2- 172.21.15.145:0/948605502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00073670 0x7fbe0010cb70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 -- 172.21.15.145:0/948605502 >> 172.21.15.145:0/948605502 conn(0x7fbe000fc3a0 msgr2=0x7fbe000fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:22.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 -- 172.21.15.145:0/948605502 shutdown_connections 2023-08-25T05:01:22.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.004+0000 7fbe051ff700 1 -- 172.21.15.145:0/948605502 wait complete. 2023-08-25T05:01:22.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 Processor -- start 2023-08-25T05:01:22.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 -- start start 2023-08-25T05:01:22.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe00073670 0x7fbe0019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 0x7fbe00198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00199150 con 0x7fbe0010d0b0 2023-08-25T05:01:22.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe001992c0 con 0x7fbe00073670 2023-08-25T05:01:22.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.005+0000 7fbe051ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00199480 con 0x7fbe00072d50 2023-08-25T05:01:22.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:47670/0 (socket says 172.21.15.145:47670) 2023-08-25T05:01:22.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 -- 172.21.15.145:0/2617967076 learned_addr learned my addr 172.21.15.145:0/2617967076 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:22.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdff59e700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 0x7fbe00198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfe59c700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe00073670 0x7fbe0019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.016 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 -- 172.21.15.145:0/2617967076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe00073670 msgr2=0x7fbe0019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe00073670 0x7fbe0019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 -- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 msgr2=0x7fbe00198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 0x7fbe00198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 -- 172.21.15.145:0/2617967076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40095f0 con 0x7fbe00072d50 2023-08-25T05:01:22.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdff59e700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 0x7fbe00198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T05:01:22.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbdfed9d700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9a0 tx=0x7fbdf000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:22.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0009790 con 0x7fbe00072d50 2023-08-25T05:01:22.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf000cbd0 con 0x7fbe00072d50 2023-08-25T05:01:22.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.006+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe00199760 con 0x7fbe00072d50 2023-08-25T05:01:22.018 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.007+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0018680 con 0x7fbe00072d50 2023-08-25T05:01:22.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.007+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe001a5a60 con 0x7fbe00072d50 2023-08-25T05:01:22.019 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.008+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe001082c0 con 0x7fbe00072d50 2023-08-25T05:01:22.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.010+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fbdf000cd40 con 0x7fbe00072d50 2023-08-25T05:01:22.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.011+0000 7fbde7fff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 0x7fbde80a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.011+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fbdf0014070 con 0x7fbe00072d50 2023-08-25T05:01:22.020 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.011+0000 7fbdfe59c700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 0x7fbde80a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.012+0000 7fbdfe59c700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 0x7fbde80a2600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe0010e5c0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:22.021 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.015+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fbdf0051e00 con 0x7fbe00072d50 2023-08-25T05:01:22.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:21 smithi145 ceph-mon[97681]: pgmap v511: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:22.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:21 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3443844645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:21 smithi162 ceph-mon[130941]: pgmap v511: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:22.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:21 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3443844645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:22.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:21 smithi183 ceph-mon[131414]: pgmap v511: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:22.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:21 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3443844645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:22.275 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.274+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe000676a0 con 0x7fbe00072d50 2023-08-25T05:01:22.276 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.275+0000 7fbde7fff700 1 -- 172.21.15.145:0/2617967076 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe000676a0 con 0x7fbe00072d50 2023-08-25T05:01:22.276 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2023-08-25T05:01:22.278 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.277+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 msgr2=0x7fbde80a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.277+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 0x7fbde80a2600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe0010e5c0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.277+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 msgr2=0x7fbe0019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.277+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000e9a0 tx=0x7fbdf000ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 shutdown_connections 2023-08-25T05:01:22.279 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe0010d0b0 0x7fbe00198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe00072d50 0x7fbe0019ea70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fbde80a0140 0x7fbde80a2600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 --2- 172.21.15.145:0/2617967076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe00073670 0x7fbe0019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.280 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 >> 172.21.15.145:0/2617967076 conn(0x7fbe000fc3a0 msgr2=0x7fbe000fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:22.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 shutdown_connections 2023-08-25T05:01:22.281 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.278+0000 7fbe051ff700 1 -- 172.21.15.145:0/2617967076 wait complete. 2023-08-25T05:01:22.299 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T05:01:22.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 -- 172.21.15.145:0/1111673564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 msgr2=0x7f7948106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 --2- 172.21.15.145:0/1111673564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f7948106b60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7944009990 tx=0x7f7944009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.538 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 -- 172.21.15.145:0/1111673564 shutdown_connections 2023-08-25T05:01:22.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 --2- 172.21.15.145:0/1111673564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948068440 0x7f794810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 --2- 172.21.15.145:0/1111673564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f7948106b60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 --2- 172.21.15.145:0/1111673564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948067a80 0x7f7948067f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 -- 172.21.15.145:0/1111673564 >> 172.21.15.145:0/1111673564 conn(0x7f79480750e0 msgr2=0x7f79480754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:22.539 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 -- 172.21.15.145:0/1111673564 shutdown_connections 2023-08-25T05:01:22.540 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.537+0000 7f794e59d700 1 -- 172.21.15.145:0/1111673564 wait complete. 2023-08-25T05:01:22.541 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.538+0000 7f794e59d700 1 Processor -- start 2023-08-25T05:01:22.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.538+0000 7f794e59d700 1 -- start start 2023-08-25T05:01:22.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.538+0000 7f794e59d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 0x7f79480ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794e59d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 0x7f79480ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794e59d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f79481a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.542 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794e59d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79481006a0 con 0x7f7948067a80 2023-08-25T05:01:22.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794e59d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79481a4de0 con 0x7f7948068440 2023-08-25T05:01:22.543 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794e59d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79481a4f20 con 0x7f7948106780 2023-08-25T05:01:22.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794d59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 0x7f79480ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 0x7f79480ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794d59b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 0x7f79480ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47164/0 (socket says 172.21.15.145:47164) 2023-08-25T05:01:22.544 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794d59b700 1 -- 172.21.15.145:0/2604218279 learned_addr learned my addr 172.21.15.145:0/2604218279 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:22.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794dd9c700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f79481a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 -- 172.21.15.145:0/2604218279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 msgr2=0x7f79481a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f79481a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 -- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 msgr2=0x7f79480ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.545 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 0x7f79480ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.539+0000 7f794cd9a700 1 -- 172.21.15.145:0/2604218279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440095f0 con 0x7f7948068440 2023-08-25T05:01:22.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f794cd9a700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 0x7f79480ffe70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:22.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800c9c0 con 0x7f7948068440 2023-08-25T05:01:22.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7938010430 con 0x7f7948068440 2023-08-25T05:01:22.546 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79481a5180 con 0x7f7948068440 2023-08-25T05:01:22.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938018680 con 0x7f7948068440 2023-08-25T05:01:22.547 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.540+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79481a5660 con 0x7f7948068440 2023-08-25T05:01:22.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.541+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7948067630 con 0x7f7948068440 2023-08-25T05:01:22.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.543+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f7938010860 con 0x7f7948068440 2023-08-25T05:01:22.548 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.545+0000 7f793e7fc700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 0x7f79340a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:22.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.545+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f7938014070 con 0x7f7948068440 2023-08-25T05:01:22.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.545+0000 7f794d59b700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 0x7f79340a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:22.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.546+0000 7f794d59b700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 0x7f79340a2620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f794400b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:22.549 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.549+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f7938051e00 con 0x7f7948068440 2023-08-25T05:01:22.839 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.838+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79481107b0 con 0x7f7948068440 2023-08-25T05:01:22.840 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.839+0000 7f793e7fc700 1 -- 172.21.15.145:0/2604218279 <== mon.2 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79380951c0 con 0x7f7948068440 2023-08-25T05:01:22.841 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T05:01:22.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 msgr2=0x7f79340a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 0x7f79340a2620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f794400b260 comp rx=0 tx=0).stop 2023-08-25T05:01:22.843 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 msgr2=0x7f79480ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 0x7f79480ffe70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f793800e9f0 tx=0x7f793800ed00 comp rx=0 tx=0).stop 2023-08-25T05:01:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 shutdown_connections 2023-08-25T05:01:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7948106780 0x7f79481a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7948067a80 0x7f79480ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f79340a0160 0x7f79340a2620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 --2- 172.21.15.145:0/2604218279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7948068440 0x7f79480ffe70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:22.845 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 >> 172.21.15.145:0/2604218279 conn(0x7f79480750e0 msgr2=0x7f79480fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:22.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 shutdown_connections 2023-08-25T05:01:22.846 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:22.842+0000 7f794e59d700 1 -- 172.21.15.145:0/2604218279 wait complete. 2023-08-25T05:01:22.863 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T05:01:23.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.112+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1793514689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 msgr2=0x7f5b5810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.114 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.112+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1793514689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5810f520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009f90 tx=0x7f5b5000b330 comp rx=0 tx=0).stop 2023-08-25T05:01:23.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1793514689 shutdown_connections 2023-08-25T05:01:23.115 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1793514689 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5810f520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1793514689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58072d10 0x7f5b580730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1793514689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58073630 0x7f5b5810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.116 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1793514689 >> 172.21.15.145:0/1793514689 conn(0x7f5b580fc3a0 msgr2=0x7f5b580fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:23.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1793514689 shutdown_connections 2023-08-25T05:01:23.117 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.113+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1793514689 wait complete. 2023-08-25T05:01:23.120 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 Processor -- start 2023-08-25T05:01:23.121 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 -- start start 2023-08-25T05:01:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58072d10 0x7f5b58198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819dd60 con 0x7f5b5810d0f0 2023-08-25T05:01:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819ded0 con 0x7f5b58072d10 2023-08-25T05:01:23.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.114+0000 7f5b5cbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819e090 con 0x7f5b58073630 2023-08-25T05:01:23.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:51222/0 (socket says 172.21.15.145:51222) 2023-08-25T05:01:23.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 -- 172.21.15.145:0/1953647232 learned_addr learned my addr 172.21.15.145:0/1953647232 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:23.123 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b5659c700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58072d10 0x7f5b58198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b56d9d700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 -- 172.21.15.145:0/1953647232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58072d10 msgr2=0x7f5b58198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.124 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58072d10 0x7f5b58198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 -- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 msgr2=0x7f5b5819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.125 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 -- 172.21.15.145:0/1953647232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480095f0 con 0x7f5b58073630 2023-08-25T05:01:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.115+0000 7f5b55d9b700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00e9a0 tx=0x7f5b4c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.116+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009790 con 0x7f5b58073630 2023-08-25T05:01:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.116+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c010430 con 0x7f5b58073630 2023-08-25T05:01:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.116+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b50009c40 con 0x7f5b58073630 2023-08-25T05:01:23.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.116+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c018680 con 0x7f5b58073630 2023-08-25T05:01:23.128 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.116+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5819e560 con 0x7f5b58073630 2023-08-25T05:01:23.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.117+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58067af0 con 0x7f5b58073630 2023-08-25T05:01:23.129 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.120+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f5b4c0105a0 con 0x7f5b58073630 2023-08-25T05:01:23.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.121+0000 7f5b477fe700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 0x7f5b400a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.131 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.121+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f5b4c014070 con 0x7f5b58073630 2023-08-25T05:01:23.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.121+0000 7f5b5659c700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 0x7f5b400a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.122+0000 7f5b5659c700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 0x7f5b400a2600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48009310 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:23.132 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.122+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f5b4c095260 con 0x7f5b58073630 2023-08-25T05:01:23.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:22 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2617967076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:22 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/2604218279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:22 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2617967076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:22 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/2604218279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:22 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2617967076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:22 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/2604218279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-25T05:01:23.375 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.375+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b58112a70 con 0x7f5b58073630 2023-08-25T05:01:23.376 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.375+0000 7f5b477fe700 1 -- 172.21.15.145:0/1953647232 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b4c098880 con 0x7f5b58073630 2023-08-25T05:01:23.376 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2023-08-25T05:01:23.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 msgr2=0x7f5b400a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 0x7f5b400a2600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48009310 comp rx=0 tx=0).stop 2023-08-25T05:01:23.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 msgr2=0x7f5b581992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00e9a0 tx=0x7f5b4c00ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.379 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 shutdown_connections 2023-08-25T05:01:23.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b5810d0f0 0x7f5b5819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f5b400a0140 0x7f5b400a2600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b58072d10 0x7f5b58198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 --2- 172.21.15.145:0/1953647232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58073630 0x7f5b581992c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.380 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 >> 172.21.15.145:0/1953647232 conn(0x7f5b580fc3a0 msgr2=0x7f5b5810f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:23.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 shutdown_connections 2023-08-25T05:01:23.381 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.378+0000 7f5b5cbff700 1 -- 172.21.15.145:0/1953647232 wait complete. 2023-08-25T05:01:23.402 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-25T05:01:23.618 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 -- 172.21.15.145:0/1211088853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8270100890 msgr2=0x7f8270100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 --2- 172.21.15.145:0/1211088853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8270100890 0x7f8270100d10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8264009990 tx=0x7f8264009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 -- 172.21.15.145:0/1211088853 shutdown_connections 2023-08-25T05:01:23.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 --2- 172.21.15.145:0/1211088853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270101250 0x7f827010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.619 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 --2- 172.21.15.145:0/1211088853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82701068a0 0x7f8270106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 --2- 172.21.15.145:0/1211088853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8270100890 0x7f8270100d10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.617+0000 7f82755ff700 1 -- 172.21.15.145:0/1211088853 >> 172.21.15.145:0/1211088853 conn(0x7f82700fc380 msgr2=0x7f82700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:23.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.618+0000 7f82755ff700 1 -- 172.21.15.145:0/1211088853 shutdown_connections 2023-08-25T05:01:23.620 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.618+0000 7f82755ff700 1 -- 172.21.15.145:0/1211088853 wait complete. 2023-08-25T05:01:23.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 Processor -- start 2023-08-25T05:01:23.622 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 -- start start 2023-08-25T05:01:23.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 0x7f8270073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 0x7f82701a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.623 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8270073990 con 0x7f8270101250 2023-08-25T05:01:23.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701a50c0 con 0x7f8270100890 2023-08-25T05:01:23.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.619+0000 7f82755ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701a5200 con 0x7f82701068a0 2023-08-25T05:01:23.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 0x7f82701a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.624 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826e7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 0x7f8270073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.145:37760/0 (socket says 172.21.15.145:37760) 2023-08-25T05:01:23.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826effd700 1 -- 172.21.15.145:0/3053345975 learned_addr learned my addr 172.21.15.145:0/3053345975 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:23.625 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 -- 172.21.15.145:0/3053345975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 msgr2=0x7f8270072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 -- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 msgr2=0x7f8270073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 0x7f8270073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.626 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 -- 172.21.15.145:0/3053345975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640095f0 con 0x7f82701068a0 2023-08-25T05:01:23.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826effd700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:23.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826e7fc700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 0x7f8270073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:23.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.620+0000 7f826f7fe700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 0x7f82701a4b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f826800efc0 tx=0x7f826800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:23.627 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.621+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268018070 con 0x7f82701068a0 2023-08-25T05:01:23.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.621+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f826800f040 con 0x7f82701068a0 2023-08-25T05:01:23.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.621+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82701a5400 con 0x7f82701068a0 2023-08-25T05:01:23.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.621+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800cd40 con 0x7f82701068a0 2023-08-25T05:01:23.628 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.621+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701a58e0 con 0x7f82701068a0 2023-08-25T05:01:23.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.622+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82700676a0 con 0x7f82701068a0 2023-08-25T05:01:23.629 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.624+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f8268003a00 con 0x7f82701068a0 2023-08-25T05:01:23.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.625+0000 7f8257fff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 0x7f82580a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:23.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.626+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f82680cb4c0 con 0x7f82701068a0 2023-08-25T05:01:23.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.626+0000 7f826effd700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 0x7f82580a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:23.630 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.626+0000 7f826effd700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 0x7f82580a2600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8260011550 tx=0x7f8260009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:23.631 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.629+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f8268094c60 con 0x7f82701068a0 2023-08-25T05:01:23.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.897+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f82701109c0 con 0x7f82701068a0 2023-08-25T05:01:23.898 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.897+0000 7f8257fff700 1 -- 172.21.15.145:0/3053345975 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v110) v1 ==== 103+0+11 (secure 0 0 0) 0x7f826801b320 con 0x7f82701068a0 2023-08-25T05:01:23.899 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2023-08-25T05:01:23.901 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 msgr2=0x7f82580a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.901 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 0x7f82580a2600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8260011550 tx=0x7f8260009260 comp rx=0 tx=0).stop 2023-08-25T05:01:23.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 msgr2=0x7f82701a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:23.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 0x7f82701a4b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f826800efc0 tx=0x7f826800c3a0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 shutdown_connections 2023-08-25T05:01:23.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82701068a0 0x7f82701a4b80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.902 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f82580a0140 0x7f82580a2600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8270100890 0x7f8270072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.900+0000 7f82755ff700 1 --2- 172.21.15.145:0/3053345975 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8270101250 0x7f8270073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:23.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.901+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 >> 172.21.15.145:0/3053345975 conn(0x7f82700fc380 msgr2=0x7f82700fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:23.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.901+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 shutdown_connections 2023-08-25T05:01:23.903 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:23.901+0000 7f82755ff700 1 -- 172.21.15.145:0/3053345975 wait complete. 2023-08-25T05:01:23.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T05:01:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:23 smithi162 ceph-mon[130941]: pgmap v512: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:24.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:23 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1953647232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:01:24.171 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 -- 172.21.15.145:0/1229083687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 msgr2=0x7fc8ac10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:24.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/1229083687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac10f960 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400b0a0 tx=0x7fc8a400b3b0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 -- 172.21.15.145:0/1229083687 shutdown_connections 2023-08-25T05:01:24.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/1229083687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac10f960 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/1229083687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac108aa0 0x7fc8ac108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.172 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/1229083687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac102a90 0x7fc8ac102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 -- 172.21.15.145:0/1229083687 >> 172.21.15.145:0/1229083687 conn(0x7fc8ac0fe580 msgr2=0x7fc8ac1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:24.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 -- 172.21.15.145:0/1229083687 shutdown_connections 2023-08-25T05:01:24.173 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.170+0000 7fc8b13ff700 1 -- 172.21.15.145:0/1229083687 wait complete. 2023-08-25T05:01:24.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:23 smithi183 ceph-mon[131414]: pgmap v512: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:24.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:23 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1953647232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:01:24.175 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.171+0000 7fc8b13ff700 1 Processor -- start 2023-08-25T05:01:24.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 -- start start 2023-08-25T05:01:24.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 0x7fc8ac19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:24.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:24.176 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac108aa0 0x7fc8ac198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:24.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac198fe0 con 0x7fc8ac102a90 2023-08-25T05:01:24.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac199150 con 0x7fc8ac108aa0 2023-08-25T05:01:24.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8b13ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac199310 con 0x7fc8ac103450 2023-08-25T05:01:24.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8aa7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:24.177 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.172+0000 7fc8aaffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 0x7fc8ac19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:24.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:51244/0 (socket says 172.21.15.145:51244) 2023-08-25T05:01:24.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 -- 172.21.15.145:0/3881613411 learned_addr learned my addr 172.21.15.145:0/3881613411 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:24.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aaffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 0x7fc8ac19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50712/0 (socket says 172.21.15.145:50712) 2023-08-25T05:01:24.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8ab7fe700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac108aa0 0x7fc8ac198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:24.178 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 -- 172.21.15.145:0/3881613411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac108aa0 msgr2=0x7fc8ac198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:24.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac108aa0 0x7fc8ac198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 -- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 msgr2=0x7fc8ac19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:24.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 0x7fc8ac19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 -- 172.21.15.145:0/3881613411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a4009d00 con 0x7fc8ac103450 2023-08-25T05:01:24.179 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8aa7fc700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9a0 tx=0x7fc8a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:24.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0009790 con 0x7fc8ac103450 2023-08-25T05:01:24.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a000cbd0 con 0x7fc8ac103450 2023-08-25T05:01:24.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac1995f0 con 0x7fc8ac103450 2023-08-25T05:01:24.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.173+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0018680 con 0x7fc8ac103450 2023-08-25T05:01:24.180 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.174+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac1a58f0 con 0x7fc8ac103450 2023-08-25T05:01:24.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.175+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac067af0 con 0x7fc8ac103450 2023-08-25T05:01:24.181 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.177+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7fc8a000cd40 con 0x7fc8ac103450 2023-08-25T05:01:24.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.178+0000 7fc893fff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 0x7fc8940a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:24.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.179+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7fc8a0014070 con 0x7fc8ac103450 2023-08-25T05:01:24.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.179+0000 7fc8aaffd700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 0x7fc8940a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:24.182 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.179+0000 7fc8aaffd700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 0x7fc8940a2580 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009be0 tx=0x7fc89c009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:24.184 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.183+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7fc8a0051e00 con 0x7fc8ac103450 2023-08-25T05:01:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:23 smithi145 ceph-mon[97681]: pgmap v512: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:24.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:23 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1953647232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-25T05:01:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.432+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8ac112ae0 con 0x7fc8ac103450 2023-08-25T05:01:24.433 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.432+0000 7fc893fff700 1 -- 172.21.15.145:0/3881613411 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) v1 ==== 81+0+957 (secure 0 0 0) 0x7fc8a0095350 con 0x7fc8ac103450 2023-08-25T05:01:24.433 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/4132549840 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.434 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/4072524950 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.434 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/891678096 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.434 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/891678096 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.434 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/4198991683 2023-08-26T04:36:33.606385+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/3264758001 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1908809275 2023-08-26T04:36:33.606385+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/459521375 2023-08-26T04:45:04.447670+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/227013713 2023-08-26T04:36:06.999890+0000 2023-08-25T05:01:24.435 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/417312367 2023-08-26T04:36:33.606385+0000 2023-08-25T05:01:24.436 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/1961008949 2023-08-26T04:36:06.999890+0000 2023-08-25T05:01:24.436 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:0/3102359690 2023-08-26T04:36:06.999890+0000 2023-08-25T05:01:24.436 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/3601102351 2023-08-26T04:36:33.606385+0000 2023-08-25T05:01:24.436 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6800/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T05:01:24.436 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145:6801/2347349168 2023-08-26T04:36:06.999890+0000 2023-08-25T05:01:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.434+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 msgr2=0x7fc8940a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.434+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 0x7fc8940a2580 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009be0 tx=0x7fc89c009260 comp rx=0 tx=0).stop 2023-08-25T05:01:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 msgr2=0x7fc8ac19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9a0 tx=0x7fc8a000ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 shutdown_connections 2023-08-25T05:01:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7fc8940a00c0 0x7fc8940a2580 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac108aa0 0x7fc8ac198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8ac102a90 0x7fc8ac19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 --2- 172.21.15.145:0/3881613411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ac103450 0x7fc8ac19ee40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 >> 172.21.15.145:0/3881613411 conn(0x7fc8ac0fe580 msgr2=0x7fc8ac1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 shutdown_connections 2023-08-25T05:01:24.440 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:24.435+0000 7fc8b13ff700 1 -- 172.21.15.145:0/3881613411 wait complete. 2023-08-25T05:01:24.440 INFO:teuthology.orchestra.run.smithi145.stderr:listed 16 entries 2023-08-25T05:01:24.454 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-25T05:01:24.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:24.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /tmp/cephfs 2023-08-25T05:01:24.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T05:01:24.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:24.619 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-08-25T05:01:24.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:24.721 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2023-08-25T05:01:24.772 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-08-25T05:01:24.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:24.774 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-08-25T05:01:24.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:24.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2023-08-25T05:01:24.967 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-25T05:01:24.968 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-08-25T05:01:24.968 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T05:01:25.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345552k,nr_inodes=4086388,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275000k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/6caa7c5be4d467883a3819655bf9ec249bdd17a39c209c8eac83e9055b73ac38/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/6caa7c5be4d467883a3819655bf9ec249bdd17a39c209c8eac83e9055b73ac38/diff,workdir=/var/lib/containers/storage/overlay/6caa7c5be4d467883a3819655bf9ec249bdd17a39c209c8eac83e9055b73ac38/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0ba8b05bd503d8463e7539a73f4a21a3a39e381aad72ae62d0f1acb3301de712/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/0ba8b05bd503d8463e7539a73f4a21a3a39e381aad72ae62d0f1acb3301de712/diff,workdir=/var/lib/containers/storage/overlay/0ba8b05bd503d8463e7539a73f4a21a3a39e381aad72ae62d0f1acb3301de712/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275000k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/78ce124f6447c3779d32542c38c8308bcb15b109cb1768b7540a7805f51aa204/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/78ce124f6447c3779d32542c38c8308bcb15b109cb1768b7540a7805f51aa204/diff,workdir=/var/lib/containers/storage/overlay/78ce124f6447c3779d32542c38c8308bcb15b109cb1768b7540a7805f51aa204/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bb75e57fd44c231f9d024730f3486a0abaaba1ddd60934fb7b8bc9fa270305e6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/bb75e57fd44c231f9d024730f3486a0abaaba1ddd60934fb7b8bc9fa270305e6/diff,workdir=/var/lib/containers/storage/overlay/bb75e57fd44c231f9d024730f3486a0abaaba1ddd60934fb7b8bc9fa270305e6/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0ec37e3adc92c593614bbf85ccbcf79d95f6e9e53ae8435ab920420e9864753d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/0ec37e3adc92c593614bbf85ccbcf79d95f6e9e53ae8435ab920420e9864753d/diff,workdir=/var/lib/containers/storage/overlay/0ec37e3adc92c593614bbf85ccbcf79d95f6e9e53ae8435ab920420e9864753d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8c5886a35031c968ae4717435f1a60186b89804fc6e8d60e1e7355110e08fc7d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/8c5886a35031c968ae4717435f1a60186b89804fc6e8d60e1e7355110e08fc7d/diff,workdir=/var/lib/containers/storage/overlay/8c5886a35031c968ae4717435f1a60186b89804fc6e8d60e1e7355110e08fc7d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3d7b99f8642c6b13722c94b98c595aae287d9fdbe515413684659d2e53f321bb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/3d7b99f8642c6b13722c94b98c595aae287d9fdbe515413684659d2e53f321bb/diff,workdir=/var/lib/containers/storage/overlay/3d7b99f8642c6b13722c94b98c595aae287d9fdbe515413684659d2e53f321bb/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/51ad6b612d0d872dfaf22632526cfc130dbb284dc5e64fd82458f603a1f9ad27/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/51ad6b612d0d872dfaf22632526cfc130dbb284dc5e64fd82458f603a1f9ad27/diff,workdir=/var/lib/containers/storage/overlay/51ad6b612d0d872dfaf22632526cfc130dbb284dc5e64fd82458f603a1f9ad27/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c119a91c6e11e268f2efa6a86515967269197992df20a01c99332c699321260f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/c119a91c6e11e268f2efa6a86515967269197992df20a01c99332c699321260f/diff,workdir=/var/lib/containers/storage/overlay/c119a91c6e11e268f2efa6a86515967269197992df20a01c99332c699321260f/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/292dd8c263e312840fa0acb498e7fd5ad353901d7794f168995dad66de7af212/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YFAFHJE7WZZZXC3U2QS74Q6OS6:/var/lib/containers/storage/overlay/l/A3F2U5VBE45ULB5MD3AKLF2EKH,upperdir=/var/lib/containers/storage/overlay/292dd8c263e312840fa0acb498e7fd5ad353901d7794f168995dad66de7af212/diff,workdir=/var/lib/containers/storage/overlay/292dd8c263e312840fa0acb498e7fd5ad353901d7794f168995dad66de7af212/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-08-25T05:01:25.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:25.028 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2023-08-25T05:01:25.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T05:01:25.076 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2023-08-25T05:01:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:24 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3053345975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:01:25.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:24 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3881613411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T05:01:25.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:24 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3053345975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:01:25.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:24 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3881613411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T05:01:25.185 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:24 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3053345975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-25T05:01:25.185 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:24 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3881613411' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-25T05:01:25.199 INFO:teuthology.orchestra.run.smithi145.stderr:Cannot find device "ceph-brx" 2023-08-25T05:01:25.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:01:25.203 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-08-25T05:01:25.214 INFO:tasks.cephadm:Teardown begin 2023-08-25T05:01:25.214 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T05:01:25.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T05:01:25.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 -- 172.21.15.145:0/2005311856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 msgr2=0x7f612010d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:25.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 --2- 172.21.15.145:0/2005311856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f612010d300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f611800b0a0 tx=0x7f611800b3b0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.445 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 -- 172.21.15.145:0/2005311856 shutdown_connections 2023-08-25T05:01:25.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 --2- 172.21.15.145:0/2005311856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f612010d300 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 --2- 172.21.15.145:0/2005311856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f6120101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 --2- 172.21.15.145:0/2005311856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 0x7f612010a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 -- 172.21.15.145:0/2005311856 >> 172.21.15.145:0/2005311856 conn(0x7f61200fb230 msgr2=0x7f61200fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:25.446 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 -- 172.21.15.145:0/2005311856 shutdown_connections 2023-08-25T05:01:25.447 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.444+0000 7f6124bff700 1 -- 172.21.15.145:0/2005311856 wait complete. 2023-08-25T05:01:25.448 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.445+0000 7f6124bff700 1 Processor -- start 2023-08-25T05:01:25.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.445+0000 7f6124bff700 1 -- start start 2023-08-25T05:01:25.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:25.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 0x7f61200ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:25.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f61201a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:25.449 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120100450 con 0x7f612010ae60 2023-08-25T05:01:25.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61201a5180 con 0x7f61201023b0 2023-08-25T05:01:25.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f6124bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61201a52c0 con 0x7f6120101a90 2023-08-25T05:01:25.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:25.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:51274/0 (socket says 172.21.15.145:51274) 2023-08-25T05:01:25.450 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611e59c700 1 -- 172.21.15.145:0/3313974068 learned_addr learned my addr 172.21.15.145:0/3313974068 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:25.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611dd9b700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 0x7f61200ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:25.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611ed9d700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f61201a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:25.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611e59c700 1 -- 172.21.15.145:0/3313974068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 msgr2=0x7f61200ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:25.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.446+0000 7f611e59c700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 0x7f61200ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.451 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f611e59c700 1 -- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 msgr2=0x7f61201a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:25.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f611e59c700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f61201a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f611e59c700 1 -- 172.21.15.145:0/3313974068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118009d00 con 0x7f6120101a90 2023-08-25T05:01:25.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f611e59c700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:25.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110009790 con 0x7f6120101a90 2023-08-25T05:01:25.452 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611000cbd0 con 0x7f6120101a90 2023-08-25T05:01:25.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110018680 con 0x7f6120101a90 2023-08-25T05:01:25.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61201a5520 con 0x7f6120101a90 2023-08-25T05:01:25.453 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.447+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61201a59b0 con 0x7f6120101a90 2023-08-25T05:01:25.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.448+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6120067710 con 0x7f6120101a90 2023-08-25T05:01:25.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.451+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f611000cd40 con 0x7f6120101a90 2023-08-25T05:01:25.454 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.452+0000 7f610f7fe700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 0x7f61080a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:25.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.453+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f6110014070 con 0x7f6120101a90 2023-08-25T05:01:25.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.453+0000 7f611dd9b700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 0x7f61080a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:25.455 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.453+0000 7f611dd9b700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 0x7f61080a2640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6120100ed0 tx=0x7f6114009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:25.457 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.457+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6110051e00 con 0x7f6120101a90 2023-08-25T05:01:25.752 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.751+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6120100c10 con 0x7f6120101a90 2023-08-25T05:01:25.753 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.752+0000 7f610f7fe700 1 -- 172.21.15.145:0/3313974068 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f61100952d0 con 0x7f6120101a90 2023-08-25T05:01:25.754 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:01:25.754 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:51:09.718471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T05:01:25.756 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.754+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 msgr2=0x7f61080a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:25.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.754+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 0x7f61080a2640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6120100ed0 tx=0x7f6114009260 comp rx=0 tx=0).stop 2023-08-25T05:01:25.757 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.754+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 msgr2=0x7f61200ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:25.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.754+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.758 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 shutdown_connections 2023-08-25T05:01:25.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f612010ae60 0x7f61201a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6120101a90 0x7f61200ff6e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f61080a0180 0x7f61080a2640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 --2- 172.21.15.145:0/3313974068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61201023b0 0x7f61200ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:25.759 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 >> 172.21.15.145:0/3313974068 conn(0x7f61200fb230 msgr2=0x7f612010cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:25.761 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 shutdown_connections 2023-08-25T05:01:25.762 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:25.755+0000 7f6124bff700 1 -- 172.21.15.145:0/3313974068 wait complete. 2023-08-25T05:01:25.762 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 35 2023-08-25T05:01:25.778 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T05:01:26.004 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 -- 172.21.15.145:0/385845577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8534072cf0 msgr2=0x7f85340730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 --2- 172.21.15.145:0/385845577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8534072cf0 0x7f85340730d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8524009990 tx=0x7f8524009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 -- 172.21.15.145:0/385845577 shutdown_connections 2023-08-25T05:01:26.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 --2- 172.21.15.145:0/385845577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f853410cf60 0x7f853410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 --2- 172.21.15.145:0/385845577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8534072cf0 0x7f85340730d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.005 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 --2- 172.21.15.145:0/385845577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534073610 0x7f853410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 -- 172.21.15.145:0/385845577 >> 172.21.15.145:0/385845577 conn(0x7f85340fc1b0 msgr2=0x7f85340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:26.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 -- 172.21.15.145:0/385845577 shutdown_connections 2023-08-25T05:01:26.006 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.003+0000 7f853939d700 1 -- 172.21.15.145:0/385845577 wait complete. 2023-08-25T05:01:26.007 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.004+0000 7f853939d700 1 Processor -- start 2023-08-25T05:01:26.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 -- start start 2023-08-25T05:01:26.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8534073610 0x7f853419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.008 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534198dc0 con 0x7f8534072cf0 2023-08-25T05:01:26.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534198f30 con 0x7f8534073610 2023-08-25T05:01:26.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f853939d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341990f0 con 0x7f853410cf60 2023-08-25T05:01:26.009 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f8533fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f8538b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f8538b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:51296/0 (socket says 172.21.15.145:51296) 2023-08-25T05:01:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f8538b9c700 1 -- 172.21.15.145:0/129836749 learned_addr learned my addr 172.21.15.145:0/129836749 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f8533fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50752/0 (socket says 172.21.15.145:50752) 2023-08-25T05:01:26.010 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.005+0000 7f85337fe700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8534073610 0x7f853419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 -- 172.21.15.145:0/129836749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 msgr2=0x7f8534198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 -- 172.21.15.145:0/129836749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8534073610 msgr2=0x7f853419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8534073610 0x7f853419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.011 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 -- 172.21.15.145:0/129836749 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240095f0 con 0x7f8534072cf0 2023-08-25T05:01:26.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8538b9c700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:26.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f8533fff700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8524000c00 tx=0x7f8524005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:26.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852401d070 con 0x7f8534072cf0 2023-08-25T05:01:26.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8524021410 con 0x7f8534072cf0 2023-08-25T05:01:26.012 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524017650 con 0x7f8534072cf0 2023-08-25T05:01:26.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.006+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8534199370 con 0x7f8534072cf0 2023-08-25T05:01:26.013 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.007+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85341a56d0 con 0x7f8534072cf0 2023-08-25T05:01:26.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.008+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534107880 con 0x7f8534072cf0 2023-08-25T05:01:26.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.010+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f8524021580 con 0x7f8534072cf0 2023-08-25T05:01:26.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.012+0000 7f85317fa700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 0x7f851c0a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.014 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.012+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f85240cccc0 con 0x7f8534072cf0 2023-08-25T05:01:26.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.012+0000 7f85337fe700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 0x7f851c0a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.015 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.013+0000 7f85337fe700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 0x7f851c0a26f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8534199bc0 tx=0x7f8528009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:26.017 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.016+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f85240d3080 con 0x7f8534072cf0 2023-08-25T05:01:26.077 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:25 smithi145 ceph-mon[97681]: pgmap v513: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:26.078 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:25 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3313974068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:25 smithi162 ceph-mon[130941]: pgmap v513: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:26.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:25 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3313974068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:26.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:25 smithi183 ceph-mon[131414]: pgmap v513: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-08-25T05:01:26.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:25 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3313974068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:26.347 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.346+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8534067a80 con 0x7f8534072cf0 2023-08-25T05:01:26.348 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.347+0000 7f85317fa700 1 -- 172.21.15.145:0/129836749 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f8524096400 con 0x7f8534072cf0 2023-08-25T05:01:26.348 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:01:26.349 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4172912052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4172912052},{"type":"v1","addr":"172.21.15.145:6835","nonce":4172912052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4182229769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4182229769},{"type":"v1","addr":"172.21.15.145:6837","nonce":4182229769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3652134728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3652134728},{"type":"v1","addr":"172.21.15.145:6839","nonce":3652134728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4170975809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4170975809},{"type":"v1","addr":"172.21.15.162:6833","nonce":4170975809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/916411904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916411904},{"type":"v1","addr":"172.21.15.183:6833","nonce":916411904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2094921807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2094921807},{"type":"v1","addr":"172.21.15.162:6835","nonce":2094921807}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-25T04:42:49.862390+0000","modified":"2023-08-25T04:51:09.718471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/557521219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":557521219},{"type":"v1","addr":"172.21.15.183:6835","nonce":557521219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/4241662550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":4241662550},{"type":"v1","addr":"172.21.15.183:6837","nonce":4241662550}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/1048214793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1048214793},{"type":"v1","addr":"172.21.15.162:6837","nonce":1048214793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.183:6839/3401840698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3401840698},{"type":"v1","addr":"172.21.15.183:6839","nonce":3401840698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.162:6839/2860621511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2860621511},{"type":"v1","addr":"172.21.15.162:6839","nonce":2860621511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6841/3986940142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6840","nonce":3986940142},{"type":"v1","addr":"172.21.15.145:6841","nonce":3986940142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T05:01:26.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 msgr2=0x7f851c0a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 0x7f851c0a26f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8534199bc0 tx=0x7f8528009260 comp rx=0 tx=0).stop 2023-08-25T05:01:26.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 msgr2=0x7f853419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8524000c00 tx=0x7f8524005740 comp rx=0 tx=0).stop 2023-08-25T05:01:26.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 shutdown_connections 2023-08-25T05:01:26.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853410cf60 0x7f8534198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.352 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8534072cf0 0x7f853419e6e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f851c0a0230 0x7f851c0a26f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 --2- 172.21.15.145:0/129836749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8534073610 0x7f853419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 >> 172.21.15.145:0/129836749 conn(0x7f85340fc1b0 msgr2=0x7f85341049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:26.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 shutdown_connections 2023-08-25T05:01:26.354 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.350+0000 7f853939d700 1 -- 172.21.15.145:0/129836749 wait complete. 2023-08-25T05:01:26.355 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 35 2023-08-25T05:01:26.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T05:01:26.590 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 -- 172.21.15.145:0/2684894915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa30102350 msgr2=0x7faa3010a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 --2- 172.21.15.145:0/2684894915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa30102350 0x7faa3010a830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa24009990 tx=0x7faa24009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 -- 172.21.15.145:0/2684894915 shutdown_connections 2023-08-25T05:01:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 --2- 172.21.15.145:0/2684894915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3010ae50 0x7faa3010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 --2- 172.21.15.145:0/2684894915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30101a30 0x7faa30101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 --2- 172.21.15.145:0/2684894915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa30102350 0x7faa3010a830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 -- 172.21.15.145:0/2684894915 >> 172.21.15.145:0/2684894915 conn(0x7faa300fb230 msgr2=0x7faa300fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 -- 172.21.15.145:0/2684894915 shutdown_connections 2023-08-25T05:01:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.589+0000 7faa359ff700 1 -- 172.21.15.145:0/2684894915 wait complete. 2023-08-25T05:01:26.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.590+0000 7faa359ff700 1 Processor -- start 2023-08-25T05:01:26.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 -- start start 2023-08-25T05:01:26.594 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa30101a30 0x7faa3019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 0x7faa3019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa301990e0 con 0x7faa30102350 2023-08-25T05:01:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa30199250 con 0x7faa30101a30 2023-08-25T05:01:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa359ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa30199410 con 0x7faa3010ae50 2023-08-25T05:01:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa2f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa2e7fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 0x7faa3019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa2f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.145:51316/0 (socket says 172.21.15.145:51316) 2023-08-25T05:01:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa2f7fe700 1 -- 172.21.15.145:0/4239780233 learned_addr learned my addr 172.21.15.145:0/4239780233 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.591+0000 7faa2effd700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa30101a30 0x7faa3019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 -- 172.21.15.145:0/4239780233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa30101a30 msgr2=0x7faa3019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa30101a30 0x7faa3019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 -- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 msgr2=0x7faa3019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 0x7faa3019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 -- 172.21.15.145:0/4239780233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa240095f0 con 0x7faa3010ae50 2023-08-25T05:01:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2e7fc700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 0x7faa3019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa2f7fe700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa28007ee0 tx=0x7faa2800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28017070 con 0x7faa3010ae50 2023-08-25T05:01:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2800f040 con 0x7faa3010ae50 2023-08-25T05:01:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa301996f0 con 0x7faa3010ae50 2023-08-25T05:01:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28005580 con 0x7faa3010ae50 2023-08-25T05:01:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.592+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa301a59f0 con 0x7faa3010ae50 2023-08-25T05:01:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.594+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa30067710 con 0x7faa3010ae50 2023-08-25T05:01:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.596+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7faa2800db20 con 0x7faa3010ae50 2023-08-25T05:01:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.598+0000 7faa349fd700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 0x7faa180a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.598+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7faa280caff0 con 0x7faa3010ae50 2023-08-25T05:01:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.598+0000 7faa2effd700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 0x7faa180a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:26.601 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.599+0000 7faa2effd700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 0x7faa180a2640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faa20009c60 tx=0x7faa20009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:26.603 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.602+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7faa28094700 con 0x7faa3010ae50 2023-08-25T05:01:26.879 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.878+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa301108a0 con 0x7faa3010ae50 2023-08-25T05:01:26.884 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.883+0000 7faa349fd700 1 -- 172.21.15.145:0/4239780233 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+25016 (secure 0 0 0) 0x7faa28097d20 con 0x7faa3010ae50 2023-08-25T05:01:26.885 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-08-25T05:01:26.885 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":110,"fsid":"b24d70d4-4300-11ee-9b3d-001a4aab830c","created":"2023-08-25T04:35:34.328182+0000","modified":"2023-08-25T05:01:14.845060+0000","last_up_change":"2023-08-25T04:42:03.537793+0000","last_in_change":"2023-08-25T04:41:46.197524+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T04:38:48.217192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-25T04:42:48.249947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-25T04:42:49.283219+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-25T04:42:52.807279+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6b0e0c3-39e3-4870-b353-a2e7c3811368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6803","nonce":3430955976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6805","nonce":3430955976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6809","nonce":3430955976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3430955976},{"type":"v1","addr":"172.21.15.145:6807","nonce":3430955976}]},"public_addr":"172.21.15.145:6803/3430955976","cluster_addr":"172.21.15.145:6805/3430955976","heartbeat_back_addr":"172.21.15.145:6809/3430955976","heartbeat_front_addr":"172.21.15.145:6807/3430955976","state":["exists","up"]},{"osd":1,"uuid":"825a87ec-1ec6-46eb-9e65-2aed8e37f8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6801","nonce":4196277224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6803","nonce":4196277224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6807","nonce":4196277224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":4196277224},{"type":"v1","addr":"172.21.15.162:6805","nonce":4196277224}]},"public_addr":"172.21.15.162:6801/4196277224","cluster_addr":"172.21.15.162:6803/4196277224","heartbeat_back_addr":"172.21.15.162:6807/4196277224","heartbeat_front_addr":"172.21.15.162:6805/4196277224","state":["exists","up"]},{"osd":2,"uuid":"341bc527-6a20-413a-b07c-633058761811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6801","nonce":3244881082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6803","nonce":3244881082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6807","nonce":3244881082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3244881082},{"type":"v1","addr":"172.21.15.183:6805","nonce":3244881082}]},"public_addr":"172.21.15.183:6801/3244881082","cluster_addr":"172.21.15.183:6803/3244881082","heartbeat_back_addr":"172.21.15.183:6807/3244881082","heartbeat_front_addr":"172.21.15.183:6805/3244881082","state":["exists","up"]},{"osd":3,"uuid":"d77c0b06-c1f7-4af2-ab8b-77c462d5cd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6811","nonce":3785323151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6813","nonce":3785323151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6817","nonce":3785323151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3785323151},{"type":"v1","addr":"172.21.15.145:6815","nonce":3785323151}]},"public_addr":"172.21.15.145:6811/3785323151","cluster_addr":"172.21.15.145:6813/3785323151","heartbeat_back_addr":"172.21.15.145:6817/3785323151","heartbeat_front_addr":"172.21.15.145:6815/3785323151","state":["exists","up"]},{"osd":4,"uuid":"f547bc71-6bd5-4882-baa1-d043181e9ff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6809","nonce":739956893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6811","nonce":739956893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6815","nonce":739956893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":739956893},{"type":"v1","addr":"172.21.15.162:6813","nonce":739956893}]},"public_addr":"172.21.15.162:6809/739956893","cluster_addr":"172.21.15.162:6811/739956893","heartbeat_back_addr":"172.21.15.162:6815/739956893","heartbeat_front_addr":"172.21.15.162:6813/739956893","state":["exists","up"]},{"osd":5,"uuid":"bffd6e2e-e1b7-4dee-8a67-98c02dc1162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6809","nonce":15251435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6811","nonce":15251435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6815","nonce":15251435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":15251435},{"type":"v1","addr":"172.21.15.183:6813","nonce":15251435}]},"public_addr":"172.21.15.183:6809/15251435","cluster_addr":"172.21.15.183:6811/15251435","heartbeat_back_addr":"172.21.15.183:6815/15251435","heartbeat_front_addr":"172.21.15.183:6813/15251435","state":["exists","up"]},{"osd":6,"uuid":"46c86644-5bd6-4012-8847-8e5c35e97f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6819","nonce":141129064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6821","nonce":141129064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6825","nonce":141129064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":141129064},{"type":"v1","addr":"172.21.15.145:6823","nonce":141129064}]},"public_addr":"172.21.15.145:6819/141129064","cluster_addr":"172.21.15.145:6821/141129064","heartbeat_back_addr":"172.21.15.145:6825/141129064","heartbeat_front_addr":"172.21.15.145:6823/141129064","state":["exists","up"]},{"osd":7,"uuid":"94da753b-5dbe-4a36-90c9-6d8d68d9d70b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6817","nonce":3155030948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6819","nonce":3155030948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6823","nonce":3155030948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3155030948},{"type":"v1","addr":"172.21.15.162:6821","nonce":3155030948}]},"public_addr":"172.21.15.162:6817/3155030948","cluster_addr":"172.21.15.162:6819/3155030948","heartbeat_back_addr":"172.21.15.162:6823/3155030948","heartbeat_front_addr":"172.21.15.162:6821/3155030948","state":["exists","up"]},{"osd":8,"uuid":"577926f0-d4e5-4c92-9817-38ba81609f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6817","nonce":3911543971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6819","nonce":3911543971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6823","nonce":3911543971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3911543971},{"type":"v1","addr":"172.21.15.183:6821","nonce":3911543971}]},"public_addr":"172.21.15.183:6817/3911543971","cluster_addr":"172.21.15.183:6819/3911543971","heartbeat_back_addr":"172.21.15.183:6823/3911543971","heartbeat_front_addr":"172.21.15.183:6821/3911543971","state":["exists","up"]},{"osd":9,"uuid":"0fc38b1d-c461-49db-a3cc-64ba6780cf5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6827","nonce":4228444400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6829","nonce":4228444400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6833","nonce":4228444400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":4228444400},{"type":"v1","addr":"172.21.15.145:6831","nonce":4228444400}]},"public_addr":"172.21.15.145:6827/4228444400","cluster_addr":"172.21.15.145:6829/4228444400","heartbeat_back_addr":"172.21.15.145:6833/4228444400","heartbeat_front_addr":"172.21.15.145:6831/4228444400","state":["exists","up"]},{"osd":10,"uuid":"a0aa5c31-938d-470f-bbba-9d2237cddae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6825","nonce":3093586037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6827","nonce":3093586037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6831","nonce":3093586037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3093586037},{"type":"v1","addr":"172.21.15.162:6829","nonce":3093586037}]},"public_addr":"172.21.15.162:6825/3093586037","cluster_addr":"172.21.15.162:6827/3093586037","heartbeat_back_addr":"172.21.15.162:6831/3093586037","heartbeat_front_addr":"172.21.15.162:6829/3093586037","state":["exists","up"]},{"osd":11,"uuid":"b6c06dbb-0c3c-4b63-827a-1bc6dabf1e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6825","nonce":1108166266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6827","nonce":1108166266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6831","nonce":1108166266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1108166266},{"type":"v1","addr":"172.21.15.183:6829","nonce":1108166266}]},"public_addr":"172.21.15.183:6825/1108166266","cluster_addr":"172.21.15.183:6827/1108166266","heartbeat_back_addr":"172.21.15.183:6831/1108166266","heartbeat_front_addr":"172.21.15.183:6829/1108166266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:04.316741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:25.222690+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:38:45.338689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:10.180246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:31.170085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:39:51.021808+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:15.831835+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:36.438773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:40:56.819874+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:21.153546+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:41:41.760213+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T04:42:01.782769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:0/4132549840":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4072524950":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6800/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/891678096":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/4198991683":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/3264758001":"2023-08-26T04:45:04.447670+0000","172.21.15.145:0/1908809275":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/459521375":"2023-08-26T04:45:04.447670+0000","172.21.15.145:6801/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/227013713":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/417312367":"2023-08-26T04:36:33.606385+0000","172.21.15.145:0/1961008949":"2023-08-26T04:36:06.999890+0000","172.21.15.145:0/3102359690":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6800/3601102351":"2023-08-26T04:36:33.606385+0000","172.21.15.145:6800/2347349168":"2023-08-26T04:36:06.999890+0000","172.21.15.145:6801/2347349168":"2023-08-26T04:36:06.999890+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T05:01:26.887 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.886+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 msgr2=0x7faa180a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.886+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 0x7faa180a2640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faa20009c60 tx=0x7faa20009260 comp rx=0 tx=0).stop 2023-08-25T05:01:26.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 msgr2=0x7faa30198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:26.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa28007ee0 tx=0x7faa2800d3b0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.888 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 shutdown_connections 2023-08-25T05:01:26.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3010ae50 0x7faa30198a70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7faa180a0180 0x7faa180a2640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa30101a30 0x7faa3019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 --2- 172.21.15.145:0/4239780233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa30102350 0x7faa3019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:26.889 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 >> 172.21.15.145:0/4239780233 conn(0x7faa300fb230 msgr2=0x7faa3010d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:26.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 shutdown_connections 2023-08-25T05:01:26.890 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:26.887+0000 7faa359ff700 1 -- 172.21.15.145:0/4239780233 wait complete. 2023-08-25T05:01:26.912 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T05:01:27.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 -- 172.21.15.145:0/2078377879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41028b0 msgr2=0x7f47e4102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 --2- 172.21.15.145:0/2078377879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41028b0 0x7f47e4102d30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009990 tx=0x7f47d4009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 -- 172.21.15.145:0/2078377879 shutdown_connections 2023-08-25T05:01:27.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 --2- 172.21.15.145:0/2078377879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e4103270 0x7f47e410f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 --2- 172.21.15.145:0/2078377879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e41088c0 0x7f47e4108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 --2- 172.21.15.145:0/2078377879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41028b0 0x7f47e4102d30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 -- 172.21.15.145:0/2078377879 >> 172.21.15.145:0/2078377879 conn(0x7f47e40fe360 msgr2=0x7f47e4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:27.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 -- 172.21.15.145:0/2078377879 shutdown_connections 2023-08-25T05:01:27.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.147+0000 7f47ea19d700 1 -- 172.21.15.145:0/2078377879 wait complete. 2023-08-25T05:01:27.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.148+0000 7f47ea19d700 1 Processor -- start 2023-08-25T05:01:27.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.148+0000 7f47ea19d700 1 -- start start 2023-08-25T05:01:27.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e4103270 0x7f47e419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 0x7f47e4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4198df0 con 0x7f47e41028b0 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4198f60 con 0x7f47e4103270 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47ea19d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4199120 con 0x7f47e41088c0 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50792/0 (socket says 172.21.15.145:50792) 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 -- 172.21.15.145:0/1214877331 learned_addr learned my addr 172.21.15.145:0/1214877331 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:27.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e999c700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 0x7f47e4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e899a700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e4103270 0x7f47e419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 -- 172.21.15.145:0/1214877331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 msgr2=0x7f47e4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 0x7f47e4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 -- 172.21.15.145:0/1214877331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e4103270 msgr2=0x7f47e419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e4103270 0x7f47e419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e919b700 1 -- 172.21.15.145:0/1214877331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40095f0 con 0x7f47e41028b0 2023-08-25T05:01:27.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.149+0000 7f47e999c700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 0x7f47e4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:27.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47e919b700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f47e000e9a0 tx=0x7f47e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:27.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0009790 con 0x7f47e41028b0 2023-08-25T05:01:27.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e4199400 con 0x7f47e41028b0 2023-08-25T05:01:27.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47e0010430 con 0x7f47e41028b0 2023-08-25T05:01:27.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0018680 con 0x7f47e41028b0 2023-08-25T05:01:27.156 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.150+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e41a5910 con 0x7f47e41028b0 2023-08-25T05:01:27.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.151+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e4101ff0 con 0x7f47e41028b0 2023-08-25T05:01:27.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.154+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f47e00105a0 con 0x7f47e41028b0 2023-08-25T05:01:27.157 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.155+0000 7f47da7fc700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 0x7f47d00a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.155+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9345+0+0 (secure 0 0 0) 0x7f47e0014070 con 0x7f47e41028b0 2023-08-25T05:01:27.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.156+0000 7f47e899a700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 0x7f47d00a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.158 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.156+0000 7f47e899a700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 0x7f47d00a2600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47e4108050 tx=0x7f47d400b260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:27.160 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.159+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f47e0051e00 con 0x7f47e41028b0 2023-08-25T05:01:27.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:26 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/129836749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:27.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:26 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/129836749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:27.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:26 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/129836749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-25T05:01:27.482 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.481+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47e404e700 con 0x7f47e41028b0 2023-08-25T05:01:27.488 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.488+0000 7f47da7fc700 1 -- 172.21.15.145:0/1214877331 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47e0095120 con 0x7f47e41028b0 2023-08-25T05:01:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 msgr2=0x7f47d00a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 0x7f47d00a2600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47e4108050 tx=0x7f47d400b260 comp rx=0 tx=0).stop 2023-08-25T05:01:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 msgr2=0x7f47e419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f47e000e9a0 tx=0x7f47e000ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 shutdown_connections 2023-08-25T05:01:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47e41088c0 0x7f47e4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47e41028b0 0x7f47e419e920 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f47d00a0140 0x7f47d00a2600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 --2- 172.21.15.145:0/1214877331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e4103270 0x7f47e419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 >> 172.21.15.145:0/1214877331 conn(0x7f47e40fe360 msgr2=0x7f47e41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 shutdown_connections 2023-08-25T05:01:27.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.491+0000 7f47ea19d700 1 -- 172.21.15.145:0/1214877331 wait complete. 2023-08-25T05:01:27.495 INFO:teuthology.orchestra.run.smithi145.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T05:01:27.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T05:01:27.720 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 -- 172.21.15.145:0/3930149237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc1068a0 msgr2=0x7f19bc106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 --2- 172.21.15.145:0/3930149237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc1068a0 0x7f19bc106c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009990 tx=0x7f19ac009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 -- 172.21.15.145:0/3930149237 shutdown_connections 2023-08-25T05:01:27.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 --2- 172.21.15.145:0/3930149237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 0x7f19bc10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 --2- 172.21.15.145:0/3930149237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc1068a0 0x7f19bc106c80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 --2- 172.21.15.145:0/3930149237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc100890 0x7f19bc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 -- 172.21.15.145:0/3930149237 >> 172.21.15.145:0/3930149237 conn(0x7f19bc0fc380 msgr2=0x7f19bc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:27.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 -- 172.21.15.145:0/3930149237 shutdown_connections 2023-08-25T05:01:27.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.719+0000 7f19c21ff700 1 -- 172.21.15.145:0/3930149237 wait complete. 2023-08-25T05:01:27.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.720+0000 7f19c21ff700 1 Processor -- start 2023-08-25T05:01:27.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.720+0000 7f19c21ff700 1 -- start start 2023-08-25T05:01:27.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 0x7f19bc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc1068a0 0x7f19bc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1991a0 con 0x7f19bc100890 2023-08-25T05:01:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc199310 con 0x7f19bc1068a0 2023-08-25T05:01:27.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19c21ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1994d0 con 0x7f19bc101250 2023-08-25T05:01:27.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50830/0 (socket says 172.21.15.145:50830) 2023-08-25T05:01:27.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 -- 172.21.15.145:0/351906373 learned_addr learned my addr 172.21.15.145:0/351906373 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:27.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19baffd700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 0x7f19bc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bbfff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc1068a0 0x7f19bc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 -- 172.21.15.145:0/351906373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 msgr2=0x7f19bc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 0x7f19bc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 -- 172.21.15.145:0/351906373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc1068a0 msgr2=0x7f19bc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:27.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc1068a0 0x7f19bc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:27.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.721+0000 7f19bb7fe700 1 -- 172.21.15.145:0/351906373 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0095f0 con 0x7f19bc100890 2023-08-25T05:01:27.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19bb7fe700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005910 tx=0x7f19ac004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:27.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac01d070 con 0x7f19bc100890 2023-08-25T05:01:27.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc199750 con 0x7f19bc100890 2023-08-25T05:01:27.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19ac022470 con 0x7f19bc100890 2023-08-25T05:01:27.728 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac017650 con 0x7f19bc100890 2023-08-25T05:01:27.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.722+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc1a5970 con 0x7f19bc100890 2023-08-25T05:01:27.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.723+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc0676a0 con 0x7f19bc100890 2023-08-25T05:01:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.725+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f19ac0225e0 con 0x7f19bc100890 2023-08-25T05:01:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.726+0000 7f19b8ff9700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 0x7f19a40a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.727+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9515+0+0 (secure 0 0 0) 0x7f19ac0ccc30 con 0x7f19bc100890 2023-08-25T05:01:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.727+0000 7f19baffd700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 0x7f19a40a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:27.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.727+0000 7f19baffd700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 0x7f19a40a2580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f19bc19a200 tx=0x7f19b0009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:27.731 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.730+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f19ac05ce00 con 0x7f19bc100890 2023-08-25T05:01:27.828 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a[97657]: 2023-08-25T05:01:27.482+0000 7fb8944fb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T05:01:27.829 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.829 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.830 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mds[132381]: -- 172.21.15.145:0/3999852294 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d937f771e0 con 0x55d937f4c800 2023-08-25T05:01:27.830 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: starting mds.a at 2023-08-25T05:01:27.889 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.890 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.890 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mds[156205]: -- 172.21.15.162:0/1192313095 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627eecb11e0 con 0x5627eec86800 2023-08-25T05:01:27.891 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: starting mds.b at 2023-08-25T05:01:27.891 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.891 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.892 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mds[155866]: -- 172.21.15.162:0/99950575 <== mon.2 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b148791e0 con 0x555b1484e800 2023-08-25T05:01:27.892 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: starting mds.h at 2023-08-25T05:01:27.893 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.893 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.894 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: starting mds.c at 2023-08-25T05:01:27.894 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.895 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.895 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mds[156906]: -- 172.21.15.183:0/1900722974 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558244a031e0 con 0x5582449d9000 2023-08-25T05:01:27.896 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: starting mds.f at 2023-08-25T05:01:27.897 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: ignoring --setuser ceph since I am not root 2023-08-25T05:01:27.897 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: ignoring --setgroup ceph since I am not root 2023-08-25T05:01:27.897 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: starting mds.i at 2023-08-25T05:01:27.982 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:27.981+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f19bc199f00 con 0x7f19bc100890 2023-08-25T05:01:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: pgmap v514: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 862 B/s wr, 8 op/s 2023-08-25T05:01:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/4239780233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:28.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T05:01:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T05:01:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T05:01:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T05:01:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T05:01:28.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:27 smithi162 ceph-mon[130941]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T05:01:28.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: pgmap v514: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 862 B/s wr, 8 op/s 2023-08-25T05:01:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/4239780233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T05:01:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T05:01:28.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T05:01:28.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T05:01:28.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T05:01:28.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:27 smithi183 ceph-mon[131414]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T05:01:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: pgmap v514: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 862 B/s wr, 8 op/s 2023-08-25T05:01:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/4239780233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-25T05:01:28.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-25T05:01:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-25T05:01:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T05:01:28.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: osdmap e111: 12 total, 12 up, 12 in 2023-08-25T05:01:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1214877331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-25T05:01:28.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:27 smithi145 ceph-mon[97681]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-08-25T05:01:28.490 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.489+0000 7f19b8ff9700 1 -- 172.21.15.145:0/351906373 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19ac0962d0 con 0x7f19bc100890 2023-08-25T05:01:28.493 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.492+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 msgr2=0x7f19a40a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:28.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.492+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 0x7f19a40a2580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f19bc19a200 tx=0x7f19b0009260 comp rx=0 tx=0).stop 2023-08-25T05:01:28.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.492+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 msgr2=0x7f19bc19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:28.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.492+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005910 tx=0x7f19ac004710 comp rx=0 tx=0).stop 2023-08-25T05:01:28.494 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 shutdown_connections 2023-08-25T05:01:28.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f19a40a00c0 0x7f19a40a2580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.495 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19bc1068a0 0x7f19bc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19bc100890 0x7f19bc19e9f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 --2- 172.21.15.145:0/351906373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19bc101250 0x7f19bc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.496 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 >> 172.21.15.145:0/351906373 conn(0x7f19bc0fc380 msgr2=0x7f19bc10bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:28.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 shutdown_connections 2023-08-25T05:01:28.497 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.493+0000 7f19c21ff700 1 -- 172.21.15.145:0/351906373 wait complete. 2023-08-25T05:01:28.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T05:01:28.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 -- 172.21.15.145:0/2087915471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 msgr2=0x7f1190100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:28.717 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 --2- 172.21.15.145:0/2087915471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190100d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 -- 172.21.15.145:0/2087915471 shutdown_connections 2023-08-25T05:01:28.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 --2- 172.21.15.145:0/2087915471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 0x7f119010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 --2- 172.21.15.145:0/2087915471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 0x7f1190106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.718 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 --2- 172.21.15.145:0/2087915471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190100d10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 -- 172.21.15.145:0/2087915471 >> 172.21.15.145:0/2087915471 conn(0x7f11900fc380 msgr2=0x7f11900fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:28.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 -- 172.21.15.145:0/2087915471 shutdown_connections 2023-08-25T05:01:28.719 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.716+0000 7f11977ff700 1 -- 172.21.15.145:0/2087915471 wait complete. 2023-08-25T05:01:28.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.717+0000 7f11977ff700 1 Processor -- start 2023-08-25T05:01:28.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.717+0000 7f11977ff700 1 -- start start 2023-08-25T05:01:28.721 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.717+0000 7f11977ff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:28.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.717+0000 7f11977ff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 0x7f1190196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:28.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f11977ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 0x7f119019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:28.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f11977ff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019b910 con 0x7f1190100890 2023-08-25T05:01:28.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f11977ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019ba80 con 0x7f11901068a0 2023-08-25T05:01:28.722 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f11977ff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019bc40 con 0x7f1190101250 2023-08-25T05:01:28.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:28.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50836/0 (socket says 172.21.15.145:50836) 2023-08-25T05:01:28.723 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f1194d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 0x7f1190196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:28.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 -- 172.21.15.145:0/3489438313 learned_addr learned my addr 172.21.15.145:0/3489438313 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:28.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f1195d9c700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 0x7f119019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:28.724 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 -- 172.21.15.145:0/3489438313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 msgr2=0x7f1190196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:28.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 0x7f1190196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 -- 172.21.15.145:0/3489438313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 msgr2=0x7f119019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:28.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 0x7f119019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:28.725 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 -- 172.21.15.145:0/3489438313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800095f0 con 0x7f1190100890 2023-08-25T05:01:28.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.718+0000 7f119559b700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f118c00e9a0 tx=0x7f118c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:28.726 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.719+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c009790 con 0x7f1190100890 2023-08-25T05:01:28.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.719+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f118c010430 con 0x7f1190100890 2023-08-25T05:01:28.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.719+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119019bf20 con 0x7f1190100890 2023-08-25T05:01:28.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.719+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c018680 con 0x7f1190100890 2023-08-25T05:01:28.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.719+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901a37d0 con 0x7f1190100890 2023-08-25T05:01:28.727 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.720+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11900676a0 con 0x7f1190100890 2023-08-25T05:01:28.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.722+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f118c0105a0 con 0x7f1190100890 2023-08-25T05:01:28.729 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.723+0000 7f11867fc700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 0x7f117c0a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:28.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.724+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9515+0+0 (secure 0 0 0) 0x7f118c014070 con 0x7f1190100890 2023-08-25T05:01:28.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.724+0000 7f1194d9a700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 0x7f117c0a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:28.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.724+0000 7f1194d9a700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 0x7f117c0a2680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1190197f80 tx=0x7f1180005fd0 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:28.730 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.727+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f118c051e00 con 0x7f1190100890 2023-08-25T05:01:28.937 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:28.936+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1190197cc0 con 0x7f1190100890 2023-08-25T05:01:29.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T05:01:29.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T05:01:29.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T05:01:29.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: Cluster is now healthy 2023-08-25T05:01:29.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6836/313309989,v1:172.21.15.183:6839/313309989] up:boot 2023-08-25T05:01:29.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6834/1849226563,v1:172.21.15.183:6837/1849226563] up:boot 2023-08-25T05:01:29.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.183:6835/4109421315,v1:172.21.15.183:6838/4109421315] up:boot 2023-08-25T05:01:29.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6836/81639375,v1:172.21.15.162:6838/81639375] up:boot 2023-08-25T05:01:29.164 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.162:6837/1392486857,v1:172.21.15.162:6839/1392486857] up:boot 2023-08-25T05:01:29.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: mds.? [v2:172.21.15.145:6840/2257382232,v1:172.21.15.145:6841/2257382232] up:boot 2023-08-25T05:01:29.165 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T05:01:29.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: fsmap 12 up:standby 2023-08-25T05:01:29.166 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T05:01:29.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T05:01:29.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T05:01:29.167 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T05:01:29.168 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T05:01:29.168 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:28 smithi162 ceph-mon[130941]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T05:01:29.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T05:01:29.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T05:01:29.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T05:01:29.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: Cluster is now healthy 2023-08-25T05:01:29.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6836/313309989,v1:172.21.15.183:6839/313309989] up:boot 2023-08-25T05:01:29.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6834/1849226563,v1:172.21.15.183:6837/1849226563] up:boot 2023-08-25T05:01:29.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.183:6835/4109421315,v1:172.21.15.183:6838/4109421315] up:boot 2023-08-25T05:01:29.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6836/81639375,v1:172.21.15.162:6838/81639375] up:boot 2023-08-25T05:01:29.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.162:6837/1392486857,v1:172.21.15.162:6839/1392486857] up:boot 2023-08-25T05:01:29.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: mds.? [v2:172.21.15.145:6840/2257382232,v1:172.21.15.145:6841/2257382232] up:boot 2023-08-25T05:01:29.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T05:01:29.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: fsmap 12 up:standby 2023-08-25T05:01:29.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T05:01:29.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T05:01:29.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T05:01:29.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T05:01:29.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T05:01:29.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:28 smithi183 ceph-mon[131414]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T05:01:29.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-25T05:01:29.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-25T05:01:29.331 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-25T05:01:29.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: Cluster is now healthy 2023-08-25T05:01:29.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6836/313309989,v1:172.21.15.183:6839/313309989] up:boot 2023-08-25T05:01:29.332 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6834/1849226563,v1:172.21.15.183:6837/1849226563] up:boot 2023-08-25T05:01:29.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.183:6835/4109421315,v1:172.21.15.183:6838/4109421315] up:boot 2023-08-25T05:01:29.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6836/81639375,v1:172.21.15.162:6838/81639375] up:boot 2023-08-25T05:01:29.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.162:6837/1392486857,v1:172.21.15.162:6839/1392486857] up:boot 2023-08-25T05:01:29.333 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: mds.? [v2:172.21.15.145:6840/2257382232,v1:172.21.15.145:6841/2257382232] up:boot 2023-08-25T05:01:29.334 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/351906373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-25T05:01:29.334 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: fsmap 12 up:standby 2023-08-25T05:01:29.334 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-25T05:01:29.335 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-25T05:01:29.335 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-25T05:01:29.335 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-25T05:01:29.337 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-25T05:01:29.337 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:28 smithi145 ceph-mon[97681]: from='mgr.15495 172.21.15.145:0/3040315417' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-25T05:01:29.908 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.908+0000 7f11867fc700 1 -- 172.21.15.145:0/3489438313 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7f118c098bb0 con 0x7f1190100890 2023-08-25T05:01:29.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 msgr2=0x7f117c0a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:29.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 0x7f117c0a2680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1190197f80 tx=0x7f1180005fd0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.913 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 msgr2=0x7f1190196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:29.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f118c00e9a0 tx=0x7f118c00ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 shutdown_connections 2023-08-25T05:01:29.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f117c0a01c0 0x7f117c0a2680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11901068a0 0x7f119019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.914 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1190100890 0x7f1190196970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 --2- 172.21.15.145:0/3489438313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190101250 0x7f1190196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:29.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 >> 172.21.15.145:0/3489438313 conn(0x7f11900fc380 msgr2=0x7f11900fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:29.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 shutdown_connections 2023-08-25T05:01:29.915 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:29.912+0000 7f11977ff700 1 -- 172.21.15.145:0/3489438313 wait complete. 2023-08-25T05:01:29.916 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' removed 2023-08-25T05:01:29.934 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T05:01:30.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 -- 172.21.15.145:0/984367217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 msgr2=0x7f685010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:30.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 --2- 172.21.15.145:0/984367217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685010d760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f684800b0a0 tx=0x7f684800b3b0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 -- 172.21.15.145:0/984367217 shutdown_connections 2023-08-25T05:01:30.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 --2- 172.21.15.145:0/984367217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685010d760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 --2- 172.21.15.145:0/984367217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68501068a0 0x7f6850106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 --2- 172.21.15.145:0/984367217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6850100890 0x7f6850100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 -- 172.21.15.145:0/984367217 >> 172.21.15.145:0/984367217 conn(0x7f68500fc380 msgr2=0x7f68500fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:30.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 -- 172.21.15.145:0/984367217 shutdown_connections 2023-08-25T05:01:30.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.139+0000 7f6856fff700 1 -- 172.21.15.145:0/984367217 wait complete. 2023-08-25T05:01:30.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.140+0000 7f6856fff700 1 Processor -- start 2023-08-25T05:01:30.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.140+0000 7f6856fff700 1 -- start start 2023-08-25T05:01:30.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:30.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:30.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68501068a0 0x7f68501988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:30.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501990a0 con 0x7f6850100890 2023-08-25T05:01:30.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850199210 con 0x7f6850101250 2023-08-25T05:01:30.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6856fff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501993d0 con 0x7f68501068a0 2023-08-25T05:01:30.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:30.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50862/0 (socket says 172.21.15.145:50862) 2023-08-25T05:01:30.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 -- 172.21.15.145:0/1772604649 learned_addr learned my addr 172.21.15.145:0/1772604649 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:30.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f684ffff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:30.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f685559c700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68501068a0 0x7f68501988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:30.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 -- 172.21.15.145:0/1772604649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68501068a0 msgr2=0x7f68501988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:30.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68501068a0 0x7f68501988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 -- 172.21.15.145:0/1772604649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 msgr2=0x7f685019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:30.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f6854d9b700 1 -- 172.21.15.145:0/1772604649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009d00 con 0x7f6850100890 2023-08-25T05:01:30.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.141+0000 7f684ffff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T05:01:30.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f6854d9b700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9a0 tx=0x7f684400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:30.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844009790 con 0x7f6850100890 2023-08-25T05:01:30.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68501996b0 con 0x7f6850100890 2023-08-25T05:01:30.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f684400cbd0 con 0x7f6850100890 2023-08-25T05:01:30.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844018680 con 0x7f6850100890 2023-08-25T05:01:30.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.142+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68501a5870 con 0x7f6850100890 2023-08-25T05:01:30.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.143+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500676a0 con 0x7f6850100890 2023-08-25T05:01:30.153 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.145+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f684400cd40 con 0x7f6850100890 2023-08-25T05:01:30.154 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.146+0000 7f684dffb700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 0x7f68400a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:30.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.146+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9019+0+0 (secure 0 0 0) 0x7f6844014070 con 0x7f6850100890 2023-08-25T05:01:30.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.146+0000 7f684ffff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 0x7f68400a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:30.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.147+0000 7f684ffff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 0x7f68400a2580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f685019a100 tx=0x7f683c009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:30.155 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.150+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f6844051e00 con 0x7f6850100890 2023-08-25T05:01:30.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:29 smithi162 ceph-mon[130941]: pgmap v516: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-08-25T05:01:30.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:29 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:30.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:29 smithi183 ceph-mon[131414]: pgmap v516: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-08-25T05:01:30.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:29 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:29 smithi145 ceph-mon[97681]: pgmap v516: 129 pgs: 129 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-08-25T05:01:30.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:29 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:30.351 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.350+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68500008d0 con 0x7f6850100890 2023-08-25T05:01:30.912 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.911+0000 7f684dffb700 1 -- 172.21.15.145:0/1772604649 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7f68440952f0 con 0x7f6850100890 2023-08-25T05:01:30.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 msgr2=0x7f68400a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:30.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 0x7f68400a2580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f685019a100 tx=0x7f683c009260 comp rx=0 tx=0).stop 2023-08-25T05:01:30.918 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 msgr2=0x7f685019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:30.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9a0 tx=0x7f684400ecb0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 shutdown_connections 2023-08-25T05:01:30.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68501068a0 0x7f68501988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.919 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6850100890 0x7f685019e8f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f68400a00c0 0x7f68400a2580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 --2- 172.21.15.145:0/1772604649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6850101250 0x7f685019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:30.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 >> 172.21.15.145:0/1772604649 conn(0x7f68500fc380 msgr2=0x7f685010bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:30.920 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 shutdown_connections 2023-08-25T05:01:30.921 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:30.916+0000 7f6856fff700 1 -- 172.21.15.145:0/1772604649 wait complete. 2023-08-25T05:01:30.922 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' removed 2023-08-25T05:01:30.937 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-25T05:01:31.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1609564001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181068a0 msgr2=0x7f3b18106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:31.140 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1609564001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181068a0 0x7f3b18106c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009990 tx=0x7f3b08009ca0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1609564001 shutdown_connections 2023-08-25T05:01:31.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1609564001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b18101250 0x7f3b1810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1609564001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b181068a0 0x7f3b18106c80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.141 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1609564001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18100890 0x7f3b18100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1609564001 >> 172.21.15.145:0/1609564001 conn(0x7f3b180fc380 msgr2=0x7f3b180fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:31.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1609564001 shutdown_connections 2023-08-25T05:01:31.142 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.139+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1609564001 wait complete. 2023-08-25T05:01:31.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.140+0000 7f3b1dbff700 1 Processor -- start 2023-08-25T05:01:31.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.140+0000 7f3b1dbff700 1 -- start start 2023-08-25T05:01:31.144 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:31.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18101250 0x7f3b1819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:31.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b181068a0 0x7f3b181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:31.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18199030 con 0x7f3b18100890 2023-08-25T05:01:31.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181991a0 con 0x7f3b18101250 2023-08-25T05:01:31.145 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b1dbff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18199360 con 0x7f3b181068a0 2023-08-25T05:01:31.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:31.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18101250 0x7f3b1819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:31.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:50880/0 (socket says 172.21.15.145:50880) 2023-08-25T05:01:31.146 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 -- 172.21.15.145:0/1602586733 learned_addr learned my addr 172.21.15.145:0/1602586733 (peer_addr_for_me v2:172.21.15.145:0/0) 2023-08-25T05:01:31.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b17fff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b181068a0 0x7f3b181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:31.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 -- 172.21.15.145:0/1602586733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b181068a0 msgr2=0x7f3b181989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:31.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b181068a0 0x7f3b181989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.147 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 -- 172.21.15.145:0/1602586733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18101250 msgr2=0x7f3b1819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:31.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18101250 0x7f3b1819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 -- 172.21.15.145:0/1602586733 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080095f0 con 0x7f3b18100890 2023-08-25T05:01:31.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.141+0000 7f3b177fe700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005910 tx=0x7f3b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:31.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.142+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0801d070 con 0x7f3b18100890 2023-08-25T05:01:31.148 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.142+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b08022470 con 0x7f3b18100890 2023-08-25T05:01:31.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.142+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08017650 con 0x7f3b18100890 2023-08-25T05:01:31.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.142+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b181995e0 con 0x7f3b18100890 2023-08-25T05:01:31.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.142+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b181a5940 con 0x7f3b18100890 2023-08-25T05:01:31.149 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.143+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b180676a0 con 0x7f3b18100890 2023-08-25T05:01:31.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.145+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 134803+0+0 (secure 0 0 0) 0x7f3b0800fad0 con 0x7f3b18100890 2023-08-25T05:01:31.150 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.147+0000 7f3b14ff9700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 0x7f3b000a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T05:01:31.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.147+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8489+0+0 (secure 0 0 0) 0x7f3b080ccb30 con 0x7f3b18100890 2023-08-25T05:01:31.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.147+0000 7f3b16ffd700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 0x7f3b000a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T05:01:31.151 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.148+0000 7f3b16ffd700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 0x7f3b000a2600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b1810dbe0 tx=0x7f3b0c009260 comp rx=0 tx=0).ready entity=mgr.15495 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T05:01:31.152 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.151+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179682 (secure 0 0 0) 0x7f3b080966f0 con 0x7f3b18100890 2023-08-25T05:01:31.161 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:30 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:31.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:30 smithi162 ceph-mon[130941]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T05:01:31.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:30 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:31.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:30 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:31.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:30 smithi183 ceph-mon[131414]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T05:01:31.174 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:30 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:31.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:30 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/3489438313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:31.329 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:30 smithi145 ceph-mon[97681]: osdmap e112: 12 total, 12 up, 12 in 2023-08-25T05:01:31.330 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:30 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:31.353 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.352+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b18199ed0 con 0x7f3b18100890 2023-08-25T05:01:31.922 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.919+0000 7f3b14ff9700 1 -- 172.21.15.145:0/1602586733 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v114) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3b08099d10 con 0x7f3b18100890 2023-08-25T05:01:31.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 msgr2=0x7f3b000a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:31.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 0x7f3b000a2600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b1810dbe0 tx=0x7f3b0c009260 comp rx=0 tx=0).stop 2023-08-25T05:01:31.924 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 msgr2=0x7f3b1819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T05:01:31.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005910 tx=0x7f3b08004730 comp rx=0 tx=0).stop 2023-08-25T05:01:31.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 shutdown_connections 2023-08-25T05:01:31.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b181068a0 0x7f3b181989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b18100890 0x7f3b1819e950 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.925 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.145:6800/2587440661,v1:172.21.15.145:6801/2587440661] conn(0x7f3b000a0140 0x7f3b000a2600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 --2- 172.21.15.145:0/1602586733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b18101250 0x7f3b1819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T05:01:31.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 >> 172.21.15.145:0/1602586733 conn(0x7f3b180fc380 msgr2=0x7f3b1810bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T05:01:31.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 shutdown_connections 2023-08-25T05:01:31.926 INFO:teuthology.orchestra.run.smithi145.stderr:2023-08-25T05:01:31.923+0000 7f3b1dbff700 1 -- 172.21.15.145:0/1602586733 wait complete. 2023-08-25T05:01:31.927 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data_ec' removed 2023-08-25T05:01:31.946 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:01:32.030 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:01:32.076 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:01:32.141 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-08-25T05:01:32.142 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T05:01:32.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:31 smithi162 ceph-mon[130941]: pgmap v518: 97 pgs: 97 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T05:01:32.162 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:31 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:32.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:31 smithi162 ceph-mon[130941]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T05:01:32.163 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:31 smithi162 ceph-mon[130941]: from='client.? 172.21.15.145:0/1602586733' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:32.172 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:31 smithi183 ceph-mon[131414]: pgmap v518: 97 pgs: 97 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T05:01:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:31 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:31 smithi183 ceph-mon[131414]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T05:01:32.173 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:31 smithi183 ceph-mon[131414]: from='client.? 172.21.15.145:0/1602586733' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:32.206 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T05:01:32.224 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-25T05:01:32.242 INFO:tasks.cephadm:Stopping all daemons... 2023-08-25T05:01:32.243 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-08-25T05:01:32.243 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a 2023-08-25T05:01:32.303 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:31 smithi145 ceph-mon[97681]: pgmap v518: 97 pgs: 97 active+clean; 6.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-25T05:01:32.303 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:31 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1772604649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-25T05:01:32.304 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:31 smithi145 ceph-mon[97681]: osdmap e113: 12 total, 12 up, 12 in 2023-08-25T05:01:32.304 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:31 smithi145 ceph-mon[97681]: from='client.? 172.21.15.145:0/1602586733' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-25T05:01:32.578 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:32 smithi145 systemd[1]: Stopping Ceph mon.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:33.073 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a[97657]: 2023-08-25T05:01:32.740+0000 7fb899d06700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:33.074 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a[97657]: 2023-08-25T05:01:32.740+0000 7fb899d06700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-08-25T05:01:33.328 INFO:journalctl@ceph.mon.a.smithi145.stdout:Aug 25 05:01:33 smithi145 bash[158781]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-a 2023-08-25T05:01:33.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.a.service' 2023-08-25T05:01:33.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:33.688 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-08-25T05:01:33.689 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-08-25T05:01:33.689 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.b 2023-08-25T05:01:34.044 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:33 smithi162 systemd[1]: Stopping Ceph mon.b for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:34.412 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-b[130917]: 2023-08-25T05:01:34.041+0000 7f1ea9ee2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:34.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-b[130917]: 2023-08-25T05:01:34.041+0000 7f1ea9ee2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-08-25T05:01:34.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.b.service' 2023-08-25T05:01:34.707 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:34 smithi162 bash[162835]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-b 2023-08-25T05:01:34.708 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:34 smithi162 systemd[1]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.b.service: Succeeded. 2023-08-25T05:01:34.708 INFO:journalctl@ceph.mon.b.smithi162.stdout:Aug 25 05:01:34 smithi162 systemd[1]: Stopped Ceph mon.b for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T05:01:34.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:34.734 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-08-25T05:01:34.734 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-08-25T05:01:34.734 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.c 2023-08-25T05:01:35.097 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:34 smithi183 systemd[1]: Stopping Ceph mon.c for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:35.098 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-c[131390]: 2023-08-25T05:01:35.095+0000 7f78cc958700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:35.098 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-c[131390]: 2023-08-25T05:01:35.095+0000 7f78cc958700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-08-25T05:01:35.736 INFO:journalctl@ceph.mon.c.smithi183.stdout:Aug 25 05:01:35 smithi183 bash[164214]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mon-c 2023-08-25T05:01:35.745 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mon.c.service' 2023-08-25T05:01:35.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:35.796 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-08-25T05:01:35.796 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-08-25T05:01:35.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x 2023-08-25T05:01:36.291 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 05:01:35 smithi145 systemd[1]: Stopping Ceph mgr.x for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:36.554 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 05:01:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T05:01:36.289+0000 7f0eab328700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:36.555 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 05:01:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x[97929]: 2023-08-25T05:01:36.289+0000 7f0eab328700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-08-25T05:01:36.828 INFO:journalctl@ceph.mgr.x.smithi145.stdout:Aug 25 05:01:36 smithi145 bash[158928]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-x 2023-08-25T05:01:37.027 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.x.service' 2023-08-25T05:01:37.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:37.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-08-25T05:01:37.117 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-08-25T05:01:37.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.y 2023-08-25T05:01:37.411 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 05:01:37 smithi162 systemd[1]: Stopping Ceph mgr.y for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:37.826 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 05:01:37 smithi162 bash[162979]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-y 2023-08-25T05:01:38.004 INFO:journalctl@ceph.mgr.y.smithi162.stdout:Aug 25 05:01:37 smithi162 systemd[1]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-08-25T05:01:38.009 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.y.service' 2023-08-25T05:01:38.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:38.047 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-08-25T05:01:38.047 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-08-25T05:01:38.047 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.z 2023-08-25T05:01:38.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 05:01:38 smithi183 systemd[1]: Stopping Ceph mgr.z for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:38.757 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 05:01:38 smithi183 bash[164356]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mgr-z 2023-08-25T05:01:38.757 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Aug 25 05:01:38 smithi183 systemd[1]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-08-25T05:01:38.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mgr.z.service' 2023-08-25T05:01:38.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:38.979 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-08-25T05:01:38.980 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-08-25T05:01:38.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.0 2023-08-25T05:01:39.328 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 05:01:39 smithi145 systemd[1]: Stopping Ceph osd.0 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:40.828 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 05:01:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T05:01:40.415+0000 7f8722e12700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:40.829 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 05:01:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T05:01:40.415+0000 7f8722e12700 -1 osd.0 114 *** Got signal Terminated *** 2023-08-25T05:01:40.829 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 05:01:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0[110738]: 2023-08-25T05:01:40.415+0000 7f8722e12700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:01:47.292 INFO:journalctl@ceph.osd.0.smithi145.stdout:Aug 25 05:01:47 smithi145 bash[159075]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-0 2023-08-25T05:01:49.124 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.0.service' 2023-08-25T05:01:49.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:49.216 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-08-25T05:01:49.217 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-08-25T05:01:49.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.1 2023-08-25T05:01:49.574 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 05:01:49 smithi162 systemd[1]: Stopping Ceph osd.1 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:49.912 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 05:01:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T05:01:49.572+0000 7f01719dc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:49.912 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 05:01:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T05:01:49.572+0000 7f01719dc700 -1 osd.1 114 *** Got signal Terminated *** 2023-08-25T05:01:49.913 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 05:01:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1[136021]: 2023-08-25T05:01:49.572+0000 7f01719dc700 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:01:55.355 INFO:journalctl@ceph.osd.1.smithi162.stdout:Aug 25 05:01:55 smithi162 bash[163122]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-1 2023-08-25T05:01:56.619 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.1.service' 2023-08-25T05:01:56.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:01:56.662 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-08-25T05:01:56.662 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-08-25T05:01:56.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.2 2023-08-25T05:01:57.026 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 05:01:56 smithi183 systemd[1]: Stopping Ceph osd.2 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:01:57.423 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 05:01:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T05:01:57.024+0000 7f92f092d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:01:57.423 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 05:01:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T05:01:57.024+0000 7f92f092d700 -1 osd.2 114 *** Got signal Terminated *** 2023-08-25T05:01:57.423 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 05:01:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2[137187]: 2023-08-25T05:01:57.024+0000 7f92f092d700 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:02.673 INFO:journalctl@ceph.osd.2.smithi183.stdout:Aug 25 05:02:02 smithi183 bash[164500]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-2 2023-08-25T05:02:03.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.2.service' 2023-08-25T05:02:04.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:04.017 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-08-25T05:02:04.017 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-08-25T05:02:04.018 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.3 2023-08-25T05:02:04.537 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 05:02:04 smithi145 systemd[1]: Stopping Ceph osd.3 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:04.828 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 05:02:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T05:02:04.533+0000 7efddb051700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:04.829 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 05:02:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T05:02:04.533+0000 7efddb051700 -1 osd.3 114 *** Got signal Terminated *** 2023-08-25T05:02:04.830 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 05:02:04 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3[116121]: 2023-08-25T05:02:04.533+0000 7efddb051700 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:07.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:07.614+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:08.913 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:08.618+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:09.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:08 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:08.804+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:09.172 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:08.912+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:09.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:09.607+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:10.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:09 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:09.799+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:10.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:09.934+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:10.776 INFO:journalctl@ceph.osd.3.smithi145.stdout:Aug 25 05:02:10 smithi145 bash[159355]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-3 2023-08-25T05:02:10.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:10.593+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:11.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:10 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:10.774+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:11.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:10 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:10.958+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:11.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:10.927+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:11.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:11 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:11.565+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:12.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:11 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:11.726+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:12.265 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.3.service' 2023-08-25T05:02:12.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:12.356 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-08-25T05:02:12.356 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-08-25T05:02:12.357 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.4 2023-08-25T05:02:12.387 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:11 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:11.961+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:12.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:11 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:11.929+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:12.661 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 05:02:12 smithi162 systemd[1]: Stopping Ceph osd.4 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:12.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:12 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:12.598+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:12.989 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 05:02:12 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T05:02:12.724+0000 7fe86aff1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:12.990 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 05:02:12 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T05:02:12.724+0000 7fe86aff1700 -1 osd.4 114 *** Got signal Terminated *** 2023-08-25T05:02:12.990 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 05:02:12 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4[141512]: 2023-08-25T05:02:12.724+0000 7fe86aff1700 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:13.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:12 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:12.721+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:13.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:12 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:12.989+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:13.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:12 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:12.975+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:13.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:13 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:13.623+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:14.077 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:13 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:13.677+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:14.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:13 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:13.994+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:14.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:13 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:13.982+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:14.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:14 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:14.596+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:15.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:14 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:14.689+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:15.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:15 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:15.035+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:15.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:15 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:15.022+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:15.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:15 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:15.574+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:16.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:15 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:15.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:16.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:16 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:16.044+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:16.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:15 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:15.974+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:16.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:16.270+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:16.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:16 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:16.615+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:16.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:16 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:16.615+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:17.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:16.676+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:17.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:17 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:17.067+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:17.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:16 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:16.945+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:17.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:16 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:16.945+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:17.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:17 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:17.266+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:17.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:17 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:17.568+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:17.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:17 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:17.568+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:18.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:17 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:17.685+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:18.411 INFO:journalctl@ceph.osd.4.smithi162.stdout:Aug 25 05:02:18 smithi162 bash[163398]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-4 2023-08-25T05:02:18.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:18 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:18.083+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:18.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:18 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:18.083+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:18.422 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:17 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:17.981+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:18.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:17 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:17.981+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:18.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:18 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:18.276+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:18.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:18 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:18.600+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:18.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:18 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:18.600+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:19.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:18 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:18.656+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:19.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:19 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:19.187+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:19.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:19 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:19.107+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:19.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:19 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:19.107+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:19.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:18 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:18.993+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:19.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:18 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:18.993+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:19.560 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.4.service' 2023-08-25T05:02:19.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:19 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:19.262+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:19.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:19.640 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-08-25T05:02:19.641 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-08-25T05:02:19.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.5 2023-08-25T05:02:19.922 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:19 smithi183 systemd[1]: Stopping Ceph osd.5 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:19.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:19 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:19.626+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:19.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:19 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:19.626+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:20.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:19 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:19.670+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:20.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:19 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:19.670+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:20.263 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:19 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:19.970+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:20.264 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:19 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:19.970+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:20.264 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:20.003+0000 7f9c7f4f7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:20.264 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:20.003+0000 7f9c7f4f7700 -1 osd.5 114 *** Got signal Terminated *** 2023-08-25T05:02:20.265 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:20.003+0000 7f9c7f4f7700 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:20.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:20 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:20.149+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:20.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:20 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:20.149+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:20.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:20 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:20.203+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:20.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:20.230+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:20.599 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:20.262+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:20.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:20.597+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:22.167 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:20 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:20.597+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:22.169 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:21 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:21.135+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:22.169 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:21 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:21.135+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:22.169 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:20.645+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:22.169 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:20.645+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:22.170 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:21 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:21.231+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:22.170 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:21 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:21.209+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:22.171 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:21.196+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:22.171 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:21 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:21.000+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:22.171 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:21 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:21.000+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:22.172 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:21 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:21.552+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:22.172 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:21 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:21.552+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:22.173 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:21.683+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:22.173 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:21 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:21.683+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:22.411 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:22 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:22.197+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:22.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:22 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:22.140+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:22.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:22 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:22.140+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:22.422 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:22 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:22.237+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:22.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:22 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:22.021+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:22.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:22 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:22.021+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:22.577 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:22 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:22.237+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:22.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:22 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:22.561+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:22.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:22 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:22.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:23.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:22 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:22.701+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:23.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:22 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:22.702+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:23.411 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:23 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:23.225+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:23.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:23 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:23.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:23.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:23 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:23.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:23.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:23 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:23.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:23.422 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:23.272+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:23.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:23.044+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:23.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:23.044+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:23.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:23.044+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:56.684622+0000 front 2023-08-25T05:01:56.684573+0000 (oldest deadline 2023-08-25T05:02:22.584750+0000) 2023-08-25T05:02:23.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:23 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:23.198+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:23.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:23.581+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:23.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:23 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:23.581+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:24.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:23 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:23.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:24.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:23 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:23.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:24.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:24 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:24.223+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:24.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:24 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:24.100+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:24.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:24 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:24.100+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:24.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:24 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:24.100+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:24.422 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:24.228+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:24.423 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:24.228+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:24.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:24.047+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.583001+0000 front 2023-08-25T05:01:42.583052+0000 (oldest deadline 2023-08-25T05:02:08.482877+0000) 2023-08-25T05:02:24.423 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:24.047+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.584222+0000 front 2023-08-25T05:01:52.584107+0000 (oldest deadline 2023-08-25T05:02:16.684212+0000) 2023-08-25T05:02:24.424 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5[142687]: 2023-08-25T05:02:24.047+0000 7f9c7a2ea700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:56.684622+0000 front 2023-08-25T05:01:56.684573+0000 (oldest deadline 2023-08-25T05:02:22.584750+0000) 2023-08-25T05:02:24.577 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:24 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:24.169+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:24.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:24 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:24.169+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:24.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:24.573+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:24.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:24.573+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:24.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:24 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:24.573+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:25.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:24 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:24.722+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:25.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:24 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:24.722+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:25.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:25.245+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:25.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:25.077+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:25.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:25.077+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:25.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:25 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:25.077+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:25.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:25.141+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:25.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:25.141+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:25.582 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:25 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:25.275+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:25.583 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:25 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:25.275+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:25.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:25 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:25.581+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:25.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:25 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:25.581+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:25.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:25 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:25.581+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:25.923 INFO:journalctl@ceph.osd.5.smithi183.stdout:Aug 25 05:02:25 smithi183 bash[164764]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-5 2023-08-25T05:02:26.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:25.698+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:26.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:25 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:25.698+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:26.411 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:26 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:26.290+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:26.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:26 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:26.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:26.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:26 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:26.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:26.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:26 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:26.121+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:26.533 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:26 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:26.307+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:26.533 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:26 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:26.307+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:26.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:26.138+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:26.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:26.138+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:26.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:26 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:26.531+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:26.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:26 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:26.531+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:26.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:26 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:26.531+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:26.955 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.5.service' 2023-08-25T05:02:26.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:26.997 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-08-25T05:02:26.998 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-08-25T05:02:26.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.6 2023-08-25T05:02:27.067 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:26.723+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:27.068 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:26.723+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:27.328 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 systemd[1]: Stopping Ceph osd.6 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:27.328 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:27.129+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:27.328 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:27.129+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:27.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:27.138+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:27.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:27.138+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:27.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:27.138+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:27.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:27.286+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:27.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:27 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:27.286+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:27.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:27.309+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:27.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:27.309+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:27.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:27.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:27.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:27.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:27.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:27 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:27.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:27.828 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:27.521+0000 7fce1ac76700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:27.829 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:27.521+0000 7fce1ac76700 -1 osd.6 114 *** Got signal Terminated *** 2023-08-25T05:02:27.829 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:27.521+0000 7fce1ac76700 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:27.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:27.722+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:27.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:27 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:27.722+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:28.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:28 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:28.241+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:28.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:28 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:28.241+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:28.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:28 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:28.111+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:28.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:28 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:28.111+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:28.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:28 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:28.111+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:28.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:28.142+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:28.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:28.142+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:28.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:28 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:28.315+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:28.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:28 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:28.315+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:28.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:28 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:28.498+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:28.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:28 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:28.498+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:28.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:28 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:28.498+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:29.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:28.713+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:29.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:28.713+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:29.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:28 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:28.713+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:29.415 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:29.264+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:29.415 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:29.264+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:29.416 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:29.123+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:29.416 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:29.123+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:29.416 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:29.123+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:29.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:29.154+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:29.579 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:29.154+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:29.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:29 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:29.314+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:29.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:29 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:29.314+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:29.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:29 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:29.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:29.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:29 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:29.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:29.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:29 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:29.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:30.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:29.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:30.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:29.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:30.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:29 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:29.703+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:30.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:30.116+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:30.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:30.116+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:30.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:30.272+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:30.663 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:30.272+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:30.663 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:30.168+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:30.663 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:30.168+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:30.664 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:30 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:30.168+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:30.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:30.347+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:30.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:30.347+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:30.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:30.485+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:30.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:30.485+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:30.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:30 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:30.485+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:31.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:30.676+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:31.304 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:30.676+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:31.304 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:30 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:30.676+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:31.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:31.141+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:31.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:31.141+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:31.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:31.141+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:31.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:31.273+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:31.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:31.273+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:31.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:31 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:31.273+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:31.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:31.107+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:31.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:31.107+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:31.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:31.367+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:31.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:31.367+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:31.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:31.367+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:31.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:31.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:31.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:31.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:31.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:31 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:31.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:32.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:31.640+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:32.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:31.640+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:32.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:31 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:31.640+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:32.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:32.108+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:32.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:32.108+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:32.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:32.108+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:32.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:32.251+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:32.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:32.251+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:32.414 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:32.251+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:32.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:32.154+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:50.004968+0000 front 2023-08-25T05:01:50.005248+0000 (oldest deadline 2023-08-25T05:02:15.305123+0000) 2023-08-25T05:02:32.578 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6[121564]: 2023-08-25T05:02:32.154+0000 7fce1726c700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:02:01.707717+0000 front 2023-08-25T05:02:01.707785+0000 (oldest deadline 2023-08-25T05:02:23.908879+0000) 2023-08-25T05:02:32.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:32.382+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:32.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:32.382+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:32.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:32.382+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:32.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:32.468+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:32.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:32.468+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:32.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:32 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:32.468+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:33.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:32.641+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:33.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:32.641+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:33.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:32.641+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:33.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:32 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:32.641+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:33.411 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:33.280+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:33.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:33.280+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:33.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:33.280+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:33.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:33.110+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:33.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:33.110+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:33.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:33.110+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:33.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:33 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:33.110+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:33.577 INFO:journalctl@ceph.osd.6.smithi145.stdout:Aug 25 05:02:33 smithi145 bash[159619]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-6 2023-08-25T05:02:33.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:33.354+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:33.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:33.354+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:33.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:33.354+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:33.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:33.510+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:33.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:33.510+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:33.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:33 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:33.510+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:34.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:33 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:33.628+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:34.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:33 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:33.628+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:34.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:33 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:33.628+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:34.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:33 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:33.628+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:34.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:34.317+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:34.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:34.317+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:34.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:34.317+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:34.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:34.078+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:34.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:34.078+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:34.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:34.078+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:34.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:34.078+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:34.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:34.309+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:34.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:34.309+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:34.675 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:34.309+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:34.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:34.464+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:34.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:34.464+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:34.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:34 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:34.464+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:34.885 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:34 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:34.600+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:34.886 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:34 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:34.600+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:34.886 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:34 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:34.600+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:34.886 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:34 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:34.600+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:35.254 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.6.service' 2023-08-25T05:02:35.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:35.344 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-08-25T05:02:35.345 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-08-25T05:02:35.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.7 2023-08-25T05:02:35.346 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:35.094+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:35.346 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:35.094+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:35.347 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:35.094+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:35.347 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:35.094+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:35.661 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.345+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:35.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.345+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:35.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.345+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:35.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 systemd[1]: Stopping Ceph osd.7 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:35.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:35.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:35.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:35.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:35.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:35.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:35.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:35.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:35.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:35.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:35.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:35 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:35.458+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:36.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:35 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:35.613+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:36.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:35 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:35.613+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:36.080 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:35 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:35.613+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:36.080 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:35 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:35.613+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:36.141 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.711+0000 7fdc9591c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:36.142 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.711+0000 7fdc9591c700 -1 osd.7 114 *** Got signal Terminated *** 2023-08-25T05:02:36.142 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:35 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:35.711+0000 7fdc9591c700 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:36.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:36.350+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:36.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:36.350+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:36.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:36.350+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:36.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:36.134+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:36.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:36.134+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:36.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:36.134+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:36.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:36 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:36.134+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:36.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:36.259+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:36.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:36.259+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:36.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:36.259+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:36.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:36.506+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:36.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:36.506+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:36.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:36 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:36.506+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:37.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:36.652+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:37.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:36.652+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:37.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:36.652+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:37.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:36 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:36.652+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:37.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:37.085+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:37.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:37.085+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:37.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:37.085+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:37.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:37.085+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:37.535 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:37.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:37.535 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:37.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:37.536 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:37.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:37.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:37.382+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:37.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:37.382+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:37.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:37 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:37.382+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:37.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:37.534+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:37.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:37.534+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:37.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:37 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:37.534+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:38.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:37.639+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:38.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:37.639+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:38.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:37.639+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:38.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:37 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:37.639+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:38.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:38.349+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:38.412 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:38.349+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:38.413 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:38.349+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:38.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:38.127+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:38.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:38.127+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:38.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:38.127+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:38.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:38 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:38.127+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:38.525 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:38.234+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:38.525 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:38.234+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:38.526 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:38.234+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:38.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:38.524+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:38.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:38.524+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:38.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:38 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:38.524+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:39.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:38 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:38.594+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:39.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:38 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:38.594+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:39.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:38 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:38.594+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:39.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:38 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:38.594+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:39.400 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:39.098+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:39.401 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:39.098+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:39.401 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:39.098+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:39.401 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:39.098+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:39.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:39.399+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:39.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:39.399+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:39.663 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:39.399+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:39.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:39.279+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:39.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:39.279+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:39.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:39.279+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:39.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:39.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:39.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:39.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:39.677 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:39.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:39.677 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:39 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:39.503+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:39.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:39 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:39.556+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:39.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:39 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:39.556+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:39.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:39 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:39.556+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:39.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:39 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:39.556+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:39.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:39 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:39.556+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:40.369 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:40.114+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:40.369 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:40.114+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:40.369 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:40.114+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:40.370 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:40.114+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:40.661 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:40.368+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:52.732224+0000 front 2023-08-25T05:01:52.732191+0000 (oldest deadline 2023-08-25T05:02:18.632395+0000) 2023-08-25T05:02:40.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:40.368+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.733577+0000 front 2023-08-25T05:01:59.733408+0000 (oldest deadline 2023-08-25T05:02:26.733926+0000) 2023-08-25T05:02:40.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:40.368+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.335879+0000 front 2023-08-25T05:02:08.335750+0000 (oldest deadline 2023-08-25T05:02:31.235652+0000) 2023-08-25T05:02:40.662 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:40 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7[147008]: 2023-08-25T05:02:40.368+0000 7fdc9070f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.136480+0000 front 2023-08-25T05:02:17.136580+0000 (oldest deadline 2023-08-25T05:02:39.436722+0000) 2023-08-25T05:02:40.672 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:40.241+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:40.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:40.242+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:40.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:40.242+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:40.673 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:40.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:40.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:40.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:40.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:40.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:40.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:40 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:40.460+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:41.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:40.588+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:41.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:40.588+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:41.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:40.588+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:41.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:40.588+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:41.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:40 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:40.588+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:41.384 INFO:journalctl@ceph.osd.7.smithi162.stdout:Aug 25 05:02:41 smithi162 bash[163658]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-7 2023-08-25T05:02:41.385 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:41.120+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:41.385 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:41.120+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:41.385 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:41.120+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:41.386 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:41 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:41.120+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:41.487 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:41.213+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:41.488 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:41.213+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:41.488 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:41.213+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:41.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:41.486+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:41.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:41.486+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:41.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:41.486+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:41.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:41 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:41.486+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:42.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:41.579+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:42.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:41.579+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:42.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:41.579+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:42.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:41.579+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:42.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:41 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:41.579+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:42.328 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:42.091+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:42.329 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:42.091+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:42.329 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:42.091+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:42.329 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:42 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:42.091+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:42.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.7.service' 2023-08-25T05:02:42.673 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:42.229+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:42.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:42.229+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:42.674 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:42.229+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:42.674 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:42.474+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:42.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:42.474+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:42.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:42.474+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:42.675 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:42.474+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:42.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:42.692 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-08-25T05:02:42.692 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-08-25T05:02:42.694 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.8 2023-08-25T05:02:42.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:42.571+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:42.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:42.571+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:42.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:42.571+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:42.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:42.571+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:42.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:42 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:42.571+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:43.042 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:42 smithi183 systemd[1]: Stopping Ceph osd.8 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:43.042 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.041+0000 7ff29df8e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:43.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:43.043+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:43.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:43.043+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:43.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:43.043+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:43.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:43 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:43.043+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:43.422 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.041+0000 7ff29df8e700 -1 osd.8 114 *** Got signal Terminated *** 2023-08-25T05:02:43.423 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.041+0000 7ff29df8e700 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:43.423 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:43.423 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:43.423 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:43.424 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:43.265+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.028687+0000 front 2023-08-25T05:02:17.028581+0000 (oldest deadline 2023-08-25T05:02:42.928503+0000) 2023-08-25T05:02:43.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:43.496+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:43.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:43.496+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:43.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:43.496+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:43.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:43 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:43.496+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:44.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:43.605+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:44.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:43.605+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:44.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:43.605+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:44.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:43.605+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:44.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:43 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:43.605+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:44.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:44.051+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:44.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:44.051+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:44.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:44.051+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:44.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:44.051+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:44.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:44 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:44.051+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:44.506 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:44.221+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:44.506 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:44.221+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:44.507 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:44.221+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:44.508 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:44.221+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.028687+0000 front 2023-08-25T05:02:17.028581+0000 (oldest deadline 2023-08-25T05:02:42.928503+0000) 2023-08-25T05:02:44.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:44.504+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:44.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:44.504+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:44.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:44.504+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:44.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:44 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:44.504+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:45.080 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:44.607+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:45.081 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:44.607+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:45.081 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:44.607+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:45.081 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:44.607+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:45.081 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:44 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:44.607+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:45.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.029+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:45.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.029+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:45.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.029+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:45.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.029+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:45.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.030+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:45.551 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:45.264+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:45.551 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:45.264+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:45.551 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:45.264+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:45.551 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:45.264+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.028687+0000 front 2023-08-25T05:02:17.028581+0000 (oldest deadline 2023-08-25T05:02:42.928503+0000) 2023-08-25T05:02:45.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:45.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:45.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:45.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:45.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:45.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:45 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:45.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:45.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:45.550+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:45.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:45.550+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:45.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:45.550+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:45.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:45 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:45.550+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:46.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.995+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:46.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.995+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:46.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.995+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:46.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.995+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:46.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:45 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:45.995+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:46.551 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:46.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:46.552 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:46.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:46.552 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:46.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:46.552 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:46.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.028687+0000 front 2023-08-25T05:02:17.028581+0000 (oldest deadline 2023-08-25T05:02:42.928503+0000) 2023-08-25T05:02:46.553 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:46.227+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.928969+0000 front 2023-08-25T05:02:22.929011+0000 (oldest deadline 2023-08-25T05:02:45.829001+0000) 2023-08-25T05:02:46.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:46.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:46.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:46.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:46.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:46.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:46 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:46.570+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:46.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:46.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:46.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:46.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:46.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:46.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:46.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:46 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:46.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:47.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:46 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:46.976+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:47.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:46 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:46.976+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:47.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:46 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:46.976+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:47.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:46 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:46.976+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:47.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:46 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:46.976+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:47.540 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:47.189+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.423879+0000 front 2023-08-25T05:01:54.424014+0000 (oldest deadline 2023-08-25T05:02:19.624486+0000) 2023-08-25T05:02:47.541 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:47.189+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.624999+0000 front 2023-08-25T05:01:59.625021+0000 (oldest deadline 2023-08-25T05:02:23.725129+0000) 2023-08-25T05:02:47.542 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:47.189+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:05.426282+0000 front 2023-08-25T05:02:05.426419+0000 (oldest deadline 2023-08-25T05:02:30.726293+0000) 2023-08-25T05:02:47.542 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:47.189+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.028687+0000 front 2023-08-25T05:02:17.028581+0000 (oldest deadline 2023-08-25T05:02:42.928503+0000) 2023-08-25T05:02:47.543 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8[148189]: 2023-08-25T05:02:47.189+0000 7ff29657c700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.928969+0000 front 2023-08-25T05:02:22.929011+0000 (oldest deadline 2023-08-25T05:02:45.829001+0000) 2023-08-25T05:02:47.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:47.540+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:47.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:47.540+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:47.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:47.540+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:47.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:47.540+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:48.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:48.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:48.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:48.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:48.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:48.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:47 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:47.604+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:48.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:48 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:48.002+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:48.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:48 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:48.002+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:48.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:48 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:48.002+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:48.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:48 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:48.002+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:48.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:48 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:48.002+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:48.729 INFO:journalctl@ceph.osd.8.smithi183.stdout:Aug 25 05:02:48 smithi183 bash[165035]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-8 2023-08-25T05:02:48.730 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:48 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:48.582+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:48.730 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:48 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:48.582+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:48.730 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:48 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:48.582+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:48.731 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:48 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:48.582+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:48.731 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:48 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:48.582+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:49.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:49.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:49.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:49.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:49.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:49.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:48 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:48.578+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:49.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:49.016+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:49.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:49.016+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:49.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:49.016+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:49.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:49.016+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:49.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:49 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:49.016+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:49.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:49.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:49.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:49.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:49.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:49.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:49 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:49.573+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:49.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:49.613+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:49.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:49.613+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:49.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:49.613+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:49.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:49.613+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:49.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:49 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:49.613+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:49.994 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.8.service' 2023-08-25T05:02:50.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:50.034 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-08-25T05:02:50.035 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-08-25T05:02:50.035 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.9 2023-08-25T05:02:50.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:50.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:50.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:50.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:50.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:50.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.004+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:50.561 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 systemd[1]: Stopping Ceph osd.9 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:50.561 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.558+0000 7fb9534d9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:50.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.558+0000 7fb9534d9700 -1 osd.9 114 *** Got signal Terminated *** 2023-08-25T05:02:50.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.558+0000 7fb9534d9700 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:50.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:50.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:50.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:50.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:50.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:50.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:50 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:50.574+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:50.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:50 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:50.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:50.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:50 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:50.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:50.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:50 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:50.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:50.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:50 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:50.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:50.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:50 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:50.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:51.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:51.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:51.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:51.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:51.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:51.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:50 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:50.974+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:51.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:51.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:51.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:51.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:51.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:51.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:51 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:51.532+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:51.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:51.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:51.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:51.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:51.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:51.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:51.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:51.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:51.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:51 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:51.608+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:52.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:52.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:52.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:52.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:52.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:52.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:51 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:51.968+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:52.833 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:52.834 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:52.834 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:52.835 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:52.836 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:52.836 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:52 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:52.554+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:52.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:52.576+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:52.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:52.576+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:52.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:52.576+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:52.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:52.576+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:52.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:52.576+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:53.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:53.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:53.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:53.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:53.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:53.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:52 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:52.966+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:53.828 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:53.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:53.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:53.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:53.829 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:53.830 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:53 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:53.564+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:53.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:53.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:53.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:53.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:53.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:53.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:53 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:53.583+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:54.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:54.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:54.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:54.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:54.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:54.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:53 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:53.983+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:54.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:54.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:54.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:54.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:54.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:54.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:54 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:54.562+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:55.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.613905+0000 front 2023-08-25T05:01:44.614078+0000 (oldest deadline 2023-08-25T05:02:08.113999+0000) 2023-08-25T05:02:55.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:53.916091+0000 front 2023-08-25T05:01:53.916206+0000 (oldest deadline 2023-08-25T05:02:19.216053+0000) 2023-08-25T05:02:55.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.216861+0000 front 2023-08-25T05:01:59.216890+0000 (oldest deadline 2023-08-25T05:02:28.017399+0000) 2023-08-25T05:02:55.078 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.518471+0000 front 2023-08-25T05:02:08.518542+0000 (oldest deadline 2023-08-25T05:02:32.618464+0000) 2023-08-25T05:02:55.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:16.020179+0000 front 2023-08-25T05:02:16.020322+0000 (oldest deadline 2023-08-25T05:02:39.519939+0000) 2023-08-25T05:02:55.079 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:54 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9[127067]: 2023-08-25T05:02:54.597+0000 7fb94d2ca700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:19.520538+0000 front 2023-08-25T05:02:19.520603+0000 (oldest deadline 2023-08-25T05:02:45.420334+0000) 2023-08-25T05:02:55.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:55.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:55.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:55.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:55.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:55.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:55.415 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:54 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:54.934+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:02:55.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:55.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:55.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:55.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:55.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:55.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:55 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:55.537+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:56.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:56.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:56.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:56.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:56.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:56.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:56.414 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:55 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:55.924+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:02:56.578 INFO:journalctl@ceph.osd.9.smithi145.stdout:Aug 25 05:02:56 smithi145 bash[159901]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-9 2023-08-25T05:02:56.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:57.977 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:57.977 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:57.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:57.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:57.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:56.528+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:57.979 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:57.980 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:57.980 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:57.980 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:57.981 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:57.981 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:57.981 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:56 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:56.914+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:02:57.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:57.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:57.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:57 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:57.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:58.253 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.9.service' 2023-08-25T05:02:58.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:02:58.345 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-08-25T05:02:58.346 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-08-25T05:02:58.346 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.10 2023-08-25T05:02:58.376 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:58.382 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:58.382 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:58.382 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:58.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:58.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:58.383 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:57 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:57.932+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:02:58.662 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 systemd[1]: Stopping Ceph osd.10 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:02:58.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:58.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:58.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:58.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:58.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:58.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:58 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:58.549+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:02:59.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.723+0000 7f5d292e7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:02:59.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.723+0000 7f5d292e7700 -1 osd.10 114 *** Got signal Terminated *** 2023-08-25T05:02:59.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.723+0000 7f5d292e7700 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:02:59.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:02:59.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:02:59.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:02:59.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:02:59.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:02:59.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:02:59.164 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:58 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:58.885+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:02:59.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:02:59.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:02:59.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:02:59.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:02:59.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:02:59.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:02:59 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:02:59.586+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:00.161 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:03:00.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:03:00.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:03:00.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:03:00.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:03:00.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:03:00.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:02:59 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:02:59.892+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:03:00.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:00.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:00.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:00.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:00.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:00.926 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:00 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:00.636+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:01.161 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:03:01.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:03:01.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:03:01.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:03:01.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:03:01.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:03:01.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:00 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:00.872+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:03:01.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:01.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:01.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:01.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:01.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:01.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:01 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:01.657+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:02.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:03:02.162 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:03:02.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:03:02.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:03:02.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:03:02.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:03:02.163 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:01 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:01.896+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:03:02.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:02.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:02.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:02.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:02.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:02.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:02 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:02.631+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:03.411 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:42.519982+0000 front 2023-08-25T05:01:42.519956+0000 (oldest deadline 2023-08-25T05:02:10.120510+0000) 2023-08-25T05:03:03.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:51.221471+0000 front 2023-08-25T05:01:51.221505+0000 (oldest deadline 2023-08-25T05:02:17.121822+0000) 2023-08-25T05:03:03.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:57.122716+0000 front 2023-08-25T05:01:57.122705+0000 (oldest deadline 2023-08-25T05:02:22.422381+0000) 2023-08-25T05:03:03.412 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6814 osd.3 since back 2023-08-25T05:02:08.323731+0000 front 2023-08-25T05:02:08.323868+0000 (oldest deadline 2023-08-25T05:02:33.023670+0000) 2023-08-25T05:03:03.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:17.525538+0000 front 2023-08-25T05:02:17.525502+0000 (oldest deadline 2023-08-25T05:02:43.425602+0000) 2023-08-25T05:03:03.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:23.426258+0000 front 2023-08-25T05:02:23.426157+0000 (oldest deadline 2023-08-25T05:02:49.326107+0000) 2023-08-25T05:03:03.413 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:02 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10[152505]: 2023-08-25T05:03:02.939+0000 7f5d230d8700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:29.326662+0000 front 2023-08-25T05:02:29.326790+0000 (oldest deadline 2023-08-25T05:02:54.626544+0000) 2023-08-25T05:03:03.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:03.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:03.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:03.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:03.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:03.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:03 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:03.622+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:04.385 INFO:journalctl@ceph.osd.10.smithi162.stdout:Aug 25 05:03:04 smithi162 bash[163922]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-10 2023-08-25T05:03:04.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:04.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:04.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:04.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:04.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:04.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:04.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:04 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:04.610+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:05.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.10.service' 2023-08-25T05:03:05.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:05.731 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-08-25T05:03:05.731 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-08-25T05:03:05.732 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.11 2023-08-25T05:03:05.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:05.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:05.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:05.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:05.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:05.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:05.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:05.624+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:05.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:05 smithi183 systemd[1]: Stopping Ceph osd.11 for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:06.422 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.086+0000 7f11314e7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:06.423 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.086+0000 7f11314e7700 -1 osd.11 114 *** Got signal Terminated *** 2023-08-25T05:03:06.423 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.086+0000 7f11314e7700 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-25T05:03:06.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:06.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:06.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:06.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:06.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:06.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:06.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:06 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:06.599+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:07.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:07.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:07.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:07.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:07.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:07.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:07.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:07 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:07.551+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:08.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:08.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:08.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:08.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:08.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:08.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:08.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:08 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:08.567+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:09.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:09.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:09.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:09.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:09.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:09.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:09.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:09 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:09.604+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:10.922 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6806 osd.0 since back 2023-08-25T05:01:44.052981+0000 front 2023-08-25T05:01:44.053061+0000 (oldest deadline 2023-08-25T05:02:06.952946+0000) 2023-08-25T05:03:10.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6804 osd.1 since back 2023-08-25T05:01:54.554875+0000 front 2023-08-25T05:01:54.554699+0000 (oldest deadline 2023-08-25T05:02:16.254775+0000) 2023-08-25T05:03:10.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2023-08-25T05:01:59.155743+0000 front 2023-08-25T05:01:59.155804+0000 (oldest deadline 2023-08-25T05:02:23.855925+0000) 2023-08-25T05:03:10.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6812 osd.4 since back 2023-08-25T05:02:15.558037+0000 front 2023-08-25T05:02:15.558144+0000 (oldest deadline 2023-08-25T05:02:39.058147+0000) 2023-08-25T05:03:10.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2023-08-25T05:02:22.559000+0000 front 2023-08-25T05:02:22.559018+0000 (oldest deadline 2023-08-25T05:02:48.459206+0000) 2023-08-25T05:03:10.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.145:6822 osd.6 since back 2023-08-25T05:02:28.459814+0000 front 2023-08-25T05:02:28.459868+0000 (oldest deadline 2023-08-25T05:02:53.159670+0000) 2023-08-25T05:03:10.924 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.162:6820 osd.7 since back 2023-08-25T05:02:38.361317+0000 front 2023-08-25T05:02:38.361458+0000 (oldest deadline 2023-08-25T05:03:03.661213+0000) 2023-08-25T05:03:10.925 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:10 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11[153723]: 2023-08-25T05:03:10.642+0000 7f112c2da700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2023-08-25T05:02:45.362075+0000 front 2023-08-25T05:02:45.362092+0000 (oldest deadline 2023-08-25T05:03:10.062094+0000) 2023-08-25T05:03:11.923 INFO:journalctl@ceph.osd.11.smithi183.stdout:Aug 25 05:03:11 smithi183 bash[165301]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-osd-11 2023-08-25T05:03:12.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@osd.11.service' 2023-08-25T05:03:12.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:12.975 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-08-25T05:03:12.975 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-08-25T05:03:12.976 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.a 2023-08-25T05:03:13.328 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:03:13 smithi145 systemd[1]: Stopping Ceph mds.a for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:13.828 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:03:13 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: 2023-08-25T05:03:13.488+0000 7ff81dad8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:13.828 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:03:13 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a[132354]: 2023-08-25T05:03:13.488+0000 7ff81dad8700 -1 mds.a *** got signal Terminated *** 2023-08-25T05:03:16.077 INFO:journalctl@ceph.mds.a.smithi145.stdout:Aug 25 05:03:15 smithi145 bash[160175]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-a 2023-08-25T05:03:16.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.a.service' 2023-08-25T05:03:16.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:16.308 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-08-25T05:03:16.309 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-08-25T05:03:16.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.d 2023-08-25T05:03:16.578 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 05:03:16 smithi145 systemd[1]: Stopping Ceph mds.d for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:17.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 05:03:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-d[131383]: 2023-08-25T05:03:16.803+0000 7f1eaa677700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:17.078 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 05:03:16 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-d[131383]: 2023-08-25T05:03:16.803+0000 7f1eaa677700 -1 mds.d *** got signal Terminated *** 2023-08-25T05:03:20.290 INFO:journalctl@ceph.mds.d.smithi145.stdout:Aug 25 05:03:19 smithi145 bash[160320]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-d 2023-08-25T05:03:20.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.d.service' 2023-08-25T05:03:20.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:20.409 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-08-25T05:03:20.410 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-08-25T05:03:20.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.g 2023-08-25T05:03:20.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 05:03:20 smithi145 systemd[1]: Stopping Ceph mds.g for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:21.328 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 05:03:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-g[131947]: 2023-08-25T05:03:20.895+0000 7f0891fc2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:21.328 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 05:03:20 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-g[131947]: 2023-08-25T05:03:20.895+0000 7f0891fc2700 -1 mds.g *** got signal Terminated *** 2023-08-25T05:03:25.578 INFO:journalctl@ceph.mds.g.smithi145.stdout:Aug 25 05:03:25 smithi145 bash[160465]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-g 2023-08-25T05:03:25.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.g.service' 2023-08-25T05:03:25.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:25.865 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-08-25T05:03:25.866 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-08-25T05:03:25.866 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.j 2023-08-25T05:03:26.077 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:25 smithi145 systemd[1]: Stopping Ceph mds.j for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:26.827 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-j[130995]: 2023-08-25T05:03:26.376+0000 7ff21503b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:26.828 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:26 smithi145 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-j[130995]: 2023-08-25T05:03:26.376+0000 7ff21503b700 -1 mds.j *** got signal Terminated *** 2023-08-25T05:03:29.078 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:28 smithi145 bash[160610]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-j 2023-08-25T05:03:29.316 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.j.service' 2023-08-25T05:03:29.376 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:29 smithi145 systemd[1]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.j.service: Succeeded. 2023-08-25T05:03:29.377 INFO:journalctl@ceph.mds.j.smithi145.stdout:Aug 25 05:03:29 smithi145 systemd[1]: Stopped Ceph mds.j for b24d70d4-4300-11ee-9b3d-001a4aab830c. 2023-08-25T05:03:29.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:29.401 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-08-25T05:03:29.401 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-08-25T05:03:29.401 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.b 2023-08-25T05:03:29.766 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:03:29 smithi162 systemd[1]: Stopping Ceph mds.b for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:30.161 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:03:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: 2023-08-25T05:03:29.765+0000 7f5db5073700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:30.162 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:03:29 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b[156181]: 2023-08-25T05:03:29.765+0000 7f5db5073700 -1 mds.b *** got signal Terminated *** 2023-08-25T05:03:32.062 INFO:journalctl@ceph.mds.b.smithi162.stdout:Aug 25 05:03:31 smithi162 bash[164194]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-b 2023-08-25T05:03:32.068 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.b.service' 2023-08-25T05:03:32.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:32.104 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-08-25T05:03:32.105 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-08-25T05:03:32.105 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.e 2023-08-25T05:03:32.412 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 05:03:32 smithi162 systemd[1]: Stopping Ceph mds.e for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:32.912 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 05:03:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-e[155499]: 2023-08-25T05:03:32.482+0000 7f6681ccd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:32.912 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 05:03:32 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-e[155499]: 2023-08-25T05:03:32.482+0000 7f6681ccd700 -1 mds.e *** got signal Terminated *** 2023-08-25T05:03:34.322 INFO:journalctl@ceph.mds.e.smithi162.stdout:Aug 25 05:03:34 smithi162 bash[164340]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-e 2023-08-25T05:03:34.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.e.service' 2023-08-25T05:03:34.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:34.365 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-08-25T05:03:34.365 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-08-25T05:03:34.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.h 2023-08-25T05:03:34.661 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:03:34 smithi162 systemd[1]: Stopping Ceph mds.h for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:35.162 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:03:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: 2023-08-25T05:03:34.739+0000 7f924b120700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:35.162 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:03:34 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h[155842]: 2023-08-25T05:03:34.739+0000 7f924b120700 -1 mds.h *** got signal Terminated *** 2023-08-25T05:03:39.162 INFO:journalctl@ceph.mds.h.smithi162.stdout:Aug 25 05:03:38 smithi162 bash[164478]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-h 2023-08-25T05:03:39.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.h.service' 2023-08-25T05:03:39.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:39.289 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-08-25T05:03:39.289 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-08-25T05:03:39.289 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.k 2023-08-25T05:03:39.661 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 05:03:39 smithi162 systemd[1]: Stopping Ceph mds.k for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:40.161 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 05:03:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-k[155159]: 2023-08-25T05:03:39.670+0000 7f58d1db7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:40.162 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 05:03:39 smithi162 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-k[155159]: 2023-08-25T05:03:39.670+0000 7f58d1db7700 -1 mds.k *** got signal Terminated *** 2023-08-25T05:03:42.133 INFO:journalctl@ceph.mds.k.smithi162.stdout:Aug 25 05:03:42 smithi162 bash[164623]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-k 2023-08-25T05:03:42.334 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.k.service' 2023-08-25T05:03:42.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:42.368 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-08-25T05:03:42.369 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-08-25T05:03:42.369 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.c 2023-08-25T05:03:42.672 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:03:42 smithi183 systemd[1]: Stopping Ceph mds.c for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:43.172 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:03:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: 2023-08-25T05:03:42.721+0000 7fe541f36700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:43.173 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:03:42 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c[157222]: 2023-08-25T05:03:42.721+0000 7fe541f36700 -1 mds.c *** got signal Terminated *** 2023-08-25T05:03:47.172 INFO:journalctl@ceph.mds.c.smithi183.stdout:Aug 25 05:03:46 smithi183 bash[165570]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-c 2023-08-25T05:03:47.246 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.c.service' 2023-08-25T05:03:47.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:47.283 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-08-25T05:03:47.284 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-08-25T05:03:47.284 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.f 2023-08-25T05:03:47.647 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:03:47 smithi183 systemd[1]: Stopping Ceph mds.f for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:47.922 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:03:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: 2023-08-25T05:03:47.645+0000 7f289b0ee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:47.923 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:03:47 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f[156882]: 2023-08-25T05:03:47.645+0000 7f289b0ee700 -1 mds.f *** got signal Terminated *** 2023-08-25T05:03:52.032 INFO:journalctl@ceph.mds.f.smithi183.stdout:Aug 25 05:03:51 smithi183 bash[165718]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-f 2023-08-25T05:03:52.038 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.f.service' 2023-08-25T05:03:52.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:52.073 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-08-25T05:03:52.074 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-08-25T05:03:52.074 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.i 2023-08-25T05:03:52.423 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:03:52 smithi183 systemd[1]: Stopping Ceph mds.i for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:52.922 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:03:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: 2023-08-25T05:03:52.452+0000 7f03b5c48700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:52.923 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:03:52 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i[157566]: 2023-08-25T05:03:52.452+0000 7f03b5c48700 -1 mds.i *** got signal Terminated *** 2023-08-25T05:03:56.105 INFO:journalctl@ceph.mds.i.smithi183.stdout:Aug 25 05:03:55 smithi183 bash[165864]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-i 2023-08-25T05:03:56.110 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.i.service' 2023-08-25T05:03:56.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:03:56.145 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-08-25T05:03:56.145 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-08-25T05:03:56.146 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.l 2023-08-25T05:03:56.422 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 05:03:56 smithi183 systemd[1]: Stopping Ceph mds.l for b24d70d4-4300-11ee-9b3d-001a4aab830c... 2023-08-25T05:03:56.922 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 05:03:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-l[156531]: 2023-08-25T05:03:56.519+0000 7f3b3e5ea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-25T05:03:56.923 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 05:03:56 smithi183 ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-l[156531]: 2023-08-25T05:03:56.519+0000 7f3b3e5ea700 -1 mds.l *** got signal Terminated *** 2023-08-25T05:03:59.992 INFO:journalctl@ceph.mds.l.smithi183.stdout:Aug 25 05:03:59 smithi183 bash[166005]: ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c-mds-l 2023-08-25T05:04:00.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b24d70d4-4300-11ee-9b3d-001a4aab830c@mds.l.service' 2023-08-25T05:04:00.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-25T05:04:00.227 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-08-25T05:04:00.228 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T05:04:00.617 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:04:03.846 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T05:04:04.151 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:04:05.717 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force --keep-logs 2023-08-25T05:04:06.016 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:04:07.572 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:04:07.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:04:07.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-25T05:04:07.717 INFO:tasks.cephadm:Archiving crash dumps... 2023-08-25T05:04:07.719 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi145/crash 2023-08-25T05:04:07.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T05:04:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T05:04:07.796 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2023-08-25T05:04:07.799 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi162/crash 2023-08-25T05:04:07.800 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T05:04:08.013 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T05:04:08.013 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2023-08-25T05:04:08.015 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi183/crash 2023-08-25T05:04:08.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash -- . 2023-08-25T05:04:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-25T05:04:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2023-08-25T05:04:08.065 INFO:tasks.cephadm:Checking cluster log for badness... 2023-08-25T05:04:08.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-08-25T05:04:08.147 INFO:teuthology.orchestra.run.smithi145.stderr:grep: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph.log: No such file or directory 2023-08-25T05:04:08.151 INFO:tasks.cephadm:Compressing logs... 2023-08-25T05:04:08.152 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T05:04:08.157 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T05:04:08.159 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T05:04:08.186 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T05:04:08.187 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2023-08-25T05:04:08.187 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.5.log 2023-08-25T05:04:08.188 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.2.log 2023-08-25T05:04:08.189 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.11.log 2023-08-25T05:04:08.189 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.5.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.z.log 2023-08-25T05:04:08.189 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.8.log 2023-08-25T05:04:08.190 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.11.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.z.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T05:04:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T05:04:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:: No such file or directory 2023-08-25T05:04:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.10.log 2023-08-25T05:04:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.h.log 2023-08-25T05:04:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.e.log 2023-08-25T05:04:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.4.log 2023-08-25T05:04:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T05:04:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.1.log 2023-08-25T05:04:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.k.log 2023-08-25T05:04:08.194 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.1.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T05:04:08.195 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.f.log 2023-08-25T05:04:08.195 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.e.log.gz 2023-08-25T05:04:08.196 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.7.log 2023-08-25T05:04:08.196 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.b.log 2023-08-25T05:04:08.197 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.c.log 2023-08-25T05:04:08.197 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.f.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.i.log 2023-08-25T05:04:08.198 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.c.log 2023-08-25T05:04:08.198 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T05:04:08.199 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.l.log 2023-08-25T05:04:08.206 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.7.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.k.log.gz 2023-08-25T05:04:08.207 INFO:teuthology.orchestra.run.smithi162.stderr: 91.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T05:04:08.207 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.b.log 2023-08-25T05:04:08.207 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.y.log 2023-08-25T05:04:08.211 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.i.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.c.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log: 91.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T05:04:08.212 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.b.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.y.log.gz 2023-08-25T05:04:08.220 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-25T05:04:08.221 INFO:teuthology.orchestra.run.smithi145.stderr:: No such file or directory 2023-08-25T05:04:08.222 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-08-25T05:04:08.223 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.0.log 2023-08-25T05:04:08.224 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.g.log 2023-08-25T05:04:08.225 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.a.log 2023-08-25T05:04:08.226 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.g.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-08-25T05:04:08.227 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.6.log 2023-08-25T05:04:08.227 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.l.log.gz 2023-08-25T05:04:08.228 INFO:teuthology.orchestra.run.smithi183.stderr: 92.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.z.log.gz 2023-08-25T05:04:08.228 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.a.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-25T05:04:08.229 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.3.log 2023-08-25T05:04:08.230 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.d.log 2023-08-25T05:04:08.230 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.g.log.gz 2023-08-25T05:04:08.232 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.x.log 2023-08-25T05:04:08.234 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.9.log 2023-08-25T05:04:08.238 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.x.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log 2023-08-25T05:04:08.239 INFO:teuthology.orchestra.run.smithi145.stderr: 94.6% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.d.log.gz 2023-08-25T05:04:08.240 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.j.log 2023-08-25T05:04:08.249 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log: /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.j.log: 94.6%gzip -5 --verbose -- /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.a.log 2023-08-25T05:04:08.250 INFO:teuthology.orchestra.run.smithi145.stderr: 91.3% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-volume.log.gz 2023-08-25T05:04:08.250 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.j.log.gz 2023-08-25T05:04:08.377 INFO:teuthology.orchestra.run.smithi183.stderr: 94.6% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.c.log.gz 2023-08-25T05:04:08.383 INFO:teuthology.orchestra.run.smithi183.stderr: 94.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.f.log.gz 2023-08-25T05:04:08.613 INFO:teuthology.orchestra.run.smithi162.stderr: 95.0% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.h.log.gz 2023-08-25T05:04:08.625 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.a.log: 89.9% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mgr.x.log.gz 2023-08-25T05:04:08.700 INFO:teuthology.orchestra.run.smithi162.stderr: 91.9% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.b.log.gz 2023-08-25T05:04:08.748 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.c.log.gz 2023-08-25T05:04:09.784 INFO:teuthology.orchestra.run.smithi145.stderr: 90.5% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mon.a.log.gz 2023-08-25T05:04:10.987 INFO:teuthology.orchestra.run.smithi162.stderr: 94.3% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.b.log.gz 2023-08-25T05:04:11.039 INFO:teuthology.orchestra.run.smithi183.stderr: 92.5% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.11.log.gz 2023-08-25T05:04:12.572 INFO:teuthology.orchestra.run.smithi162.stderr: 92.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.10.log.gz 2023-08-25T05:04:13.246 INFO:teuthology.orchestra.run.smithi162.stderr: 92.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.1.log.gz 2023-08-25T05:04:13.668 INFO:teuthology.orchestra.run.smithi145.stderr: 92.1% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.0.log.gz 2023-08-25T05:04:13.813 INFO:teuthology.orchestra.run.smithi183.stderr: 92.3% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.8.log.gz 2023-08-25T05:04:13.883 INFO:teuthology.orchestra.run.smithi162.stderr: 92.4% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.7.log.gz 2023-08-25T05:04:14.099 INFO:teuthology.orchestra.run.smithi145.stderr: 92.4% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.3.log.gz 2023-08-25T05:04:14.328 INFO:teuthology.orchestra.run.smithi162.stderr: 91.8% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.4.log.gz 2023-08-25T05:04:14.330 INFO:teuthology.orchestra.run.smithi162.stderr: 2023-08-25T05:04:14.330 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m6.154s 2023-08-25T05:04:14.330 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m24.172s 2023-08-25T05:04:14.330 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.815s 2023-08-25T05:04:14.755 INFO:teuthology.orchestra.run.smithi183.stderr: 92.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.2.log.gz 2023-08-25T05:04:16.516 INFO:teuthology.orchestra.run.smithi145.stderr: 92.4% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.6.log.gz 2023-08-25T05:04:17.207 INFO:teuthology.orchestra.run.smithi145.stderr: 92.2% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.9.log.gz 2023-08-25T05:04:17.647 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-osd.5.log.gz 2023-08-25T05:04:52.682 INFO:teuthology.orchestra.run.smithi183.stderr: 98.1% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.i.log.gz 2023-08-25T05:04:52.684 INFO:teuthology.orchestra.run.smithi183.stderr: 2023-08-25T05:04:52.685 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m44.508s 2023-08-25T05:04:52.685 INFO:teuthology.orchestra.run.smithi183.stderr:user 1m7.045s 2023-08-25T05:04:52.685 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m2.782s 2023-08-25T05:05:49.070 INFO:teuthology.orchestra.run.smithi145.stderr: 98.4% -- replaced with /var/log/ceph/b24d70d4-4300-11ee-9b3d-001a4aab830c/ceph-mds.a.log.gz 2023-08-25T05:05:49.074 INFO:teuthology.orchestra.run.smithi145.stderr: 2023-08-25T05:05:49.075 INFO:teuthology.orchestra.run.smithi145.stderr:real 1m40.880s 2023-08-25T05:05:49.075 INFO:teuthology.orchestra.run.smithi145.stderr:user 1m59.286s 2023-08-25T05:05:49.076 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m11.960s 2023-08-25T05:05:49.076 INFO:tasks.cephadm:Archiving logs... 2023-08-25T05:05:49.077 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi145/log 2023-08-25T05:05:49.078 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T05:05:56.644 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi162/log 2023-08-25T05:05:56.646 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T05:05:59.527 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi183/log 2023-08-25T05:05:59.528 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T05:06:05.197 INFO:tasks.cephadm:Removing cluster... 2023-08-25T05:06:05.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force 2023-08-25T05:06:05.587 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:06:06.507 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force 2023-08-25T05:06:07.335 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:06:07.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b24d70d4-4300-11ee-9b3d-001a4aab830c --force 2023-08-25T05:06:08.131 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: b24d70d4-4300-11ee-9b3d-001a4aab830c 2023-08-25T05:06:08.653 INFO:tasks.cephadm:Removing cephadm ... 2023-08-25T05:06:08.654 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T05:06:08.703 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T05:06:08.726 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-25T05:06:08.744 INFO:tasks.cephadm:Teardown complete 2023-08-25T05:06:08.745 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-25T05:06:08.758 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-25T05:06:08.759 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T05:06:08.764 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T05:06:08.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T05:06:08.899 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-25T05:06:08.910 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-25T05:06:08.910 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T05:06:08.915 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T05:06:08.918 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T05:06:08.934 INFO:teuthology.orchestra.run.smithi162.stderr:bash: ntpq: command not found 2023-08-25T05:06:08.934 INFO:teuthology.orchestra.run.smithi183.stderr:bash: ntpq: command not found 2023-08-25T05:06:08.939 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T05:06:08.940 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-08-25T05:06:08.940 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +1934us[+1988us] +/- 73ms 2023-08-25T05:06:08.940 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -2053us[-1999us] +/- 74ms 2023-08-25T05:06:08.940 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +196us[ +196us] +/- 74ms 2023-08-25T05:06:08.940 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T05:06:08.941 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T05:06:08.941 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-08-25T05:06:08.941 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 +1946us[+1941us] +/- 73ms 2023-08-25T05:06:08.942 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -2067us[-2072us] +/- 74ms 2023-08-25T05:06:08.942 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +191us[ +186us] +/- 74ms 2023-08-25T05:06:08.942 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T05:06:08.953 INFO:teuthology.orchestra.run.smithi145.stderr:bash: ntpq: command not found 2023-08-25T05:06:08.961 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T05:06:08.961 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2023-08-25T05:06:08.961 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 +1947us[+1947us] +/- 73ms 2023-08-25T05:06:08.962 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -2050us[-1994us] +/- 74ms 2023-08-25T05:06:08.962 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +203us[ +259us] +/- 74ms 2023-08-25T05:06:08.963 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T05:06:08.964 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-25T05:06:08.978 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-25T05:06:08.978 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-25T05:06:08.989 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T05:06:09.182 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T05:06:09.292 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T05:06:09.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T05:06:09.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:09.465 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T05:06:09.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:09.501 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T05:06:09.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:09.534 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-25T05:06:09.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-25T05:06:09.568 INFO:teuthology.task.internal:Duration was 2888.411356 seconds 2023-08-25T05:06:09.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-25T05:06:09.608 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-25T05:06:09.609 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T05:06:09.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T05:06:09.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T05:06:09.665 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T05:06:09.666 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T05:06:09.726 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T05:06:09.932 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-25T05:06:09.933 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2023-08-25T05:06:09.933 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T05:06:09.991 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2023-08-25T05:06:09.991 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T05:06:10.014 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2023-08-25T05:06:10.015 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T05:06:10.036 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-25T05:06:10.037 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T05:06:10.042 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T05:06:10.057 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T05:06:10.136 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-25T05:06:10.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T05:06:10.141 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T05:06:10.144 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T05:06:10.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-25T05:06:10.315 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-25T05:06:10.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T05:06:10.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T05:06:10.412 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T05:06:10.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-25T05:06:10.451 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.456 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.458 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.482 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2023-08-25T05:06:10.506 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2023-08-25T05:06:10.519 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2023-08-25T05:06:10.555 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:10.597 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:10.614 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T05:06:10.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T05:06:10.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-25T05:06:10.648 INFO:teuthology.task.internal:Transferring archived files... 2023-08-25T05:06:10.650 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi145 2023-08-25T05:06:10.650 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T05:06:10.730 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi162 2023-08-25T05:06:10.730 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T05:06:10.773 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064/remote/smithi183 2023-08-25T05:06:10.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T05:06:10.810 INFO:teuthology.task.internal:Removing archive directory... 2023-08-25T05:06:10.811 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T05:06:10.816 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T05:06:10.818 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T05:06:10.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-25T05:06:10.875 INFO:teuthology.task.internal:Not uploading archives. 2023-08-25T05:06:10.876 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-25T05:06:10.889 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-25T05:06:10.891 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T05:06:10.898 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T05:06:10.901 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T05:06:10.915 INFO:teuthology.orchestra.run.smithi162.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 05:06 /home/ubuntu/cephtest 2023-08-25T05:06:10.921 INFO:teuthology.orchestra.run.smithi183.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 05:06 /home/ubuntu/cephtest 2023-08-25T05:06:10.937 INFO:teuthology.orchestra.run.smithi145.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 05:06 /home/ubuntu/cephtest 2023-08-25T05:06:10.940 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-25T05:06:10.952 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-25T05:06:11.041 INFO:teuthology.nuke:Checking targets against current locks 2023-08-25T05:06:11.067 DEBUG:teuthology.nuke:shortname: smithi145 2023-08-25T05:06:11.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T05:06:11.090 DEBUG:teuthology.nuke:shortname: smithi162 2023-08-25T05:06:11.091 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T05:06:11.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.945115', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T05:06:11.119 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.948839', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T05:06:11.127 DEBUG:teuthology.nuke:shortname: smithi183 2023-08-25T05:06:11.128 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T05:06:11.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379064', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 04:08:10.947108', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T05:06:11.163 INFO:teuthology.orchestra.console.smithi145:Power off 2023-08-25T05:06:11.164 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T05:06:11.191 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2023-08-25T05:06:11.192 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:11.218 INFO:teuthology.orchestra.console.smithi162:Power off 2023-08-25T05:06:11.219 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T05:06:11.235 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2023-08-25T05:06:11.247 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-08-25T05:06:11.248 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:11.271 INFO:teuthology.orchestra.console.smithi183:Power off 2023-08-25T05:06:11.272 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T05:06:11.288 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-08-25T05:06:11.298 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-08-25T05:06:11.298 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:11.323 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-08-25T05:06:15.237 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:15.289 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:15.325 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:15.352 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2023-08-25T05:06:15.403 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-08-25T05:06:15.439 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-08-25T05:06:19.356 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:19.404 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:19.440 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:19.469 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2023-08-25T05:06:19.517 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-08-25T05:06:19.553 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-08-25T05:06:19.570 INFO:teuthology.orchestra.console.smithi145:Power off completed 2023-08-25T05:06:19.655 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-08-25T05:06:23.520 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T05:06:23.633 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-08-25T05:06:23.735 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-08-25T05:06:23.842 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} duration: 2888.4113557338715 failure_reason: 'Command failed on smithi145 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''sudo -u postgres -- pgbench -s 500 -i''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5b701f5cda6a482b84dec29cd095950d status: fail success: false 2023-08-25T05:06:23.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T05:06:23.935 INFO:teuthology.run:FAIL